2024-02-26T05:57:59.381 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-26T05:57:59.381 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} --owner scheduled_vshankar@teuthology --verbose --name vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427 -- /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/orig.config.yaml 2024-02-26T05:57:59.442 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T05:57:59.545 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427 branch: wip-vshankar-testing-20240216.060239 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} email: vshankar@redhat.com first_in_suite: false job_id: '7573427' kernel: client: branch: testing flavor: default kdb: true sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240216.060239 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240216.060239 sha1: ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2171 sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 sleep_before_teardown: 0 subset: 54/1024 suite: fs:workload suite_branch: wip-vshankar-testing-20240216.060239 suite_path: /home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa suite_relpath: qa suite_repo: https://github.com/vshankar/ceph.git suite_sha1: ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZPdkDZSRXwvoUvPcwAYAldqb0CHJhFpgKlOxRkyHNF6rbw1E2/xgFSq4EqU7IaDXqrmGdkezUdpgdkbmhtm9I= smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCWZElmQG/kXQ7G++W9esi9RidN6ruebCJTuqMKCH6GTXUldq95uZ52llHODIcBZYdrfxfYQpcyH1IpskP5OiY= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJN6FQPyU3rQNmlN7XjX9ozDAbCOMd7C4QTI8TsxNvfiK+XiywOGT8wepALDNcH6c3vpDYA/5J2Lik6ezL6wxo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-26_05:44:42 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-26T05:57:59.545 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa; will attempt to use it 2024-02-26T05:57:59.546 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks 2024-02-26T05:57:59.546 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-26T05:57:59.548 INFO:teuthology.task.internal:Checking packages... 2024-02-26T05:57:59.580 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '25296a8f176256aea5ed8b5850b25167dce0ae33' 2024-02-26T05:57:59.580 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-26T05:57:59.580 INFO:teuthology.packaging:ref: None 2024-02-26T05:57:59.580 INFO:teuthology.packaging:tag: None 2024-02-26T05:57:59.581 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240216.060239 2024-02-26T05:57:59.581 INFO:teuthology.packaging:sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T05:57:59.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240216.060239 2024-02-26T05:57:59.984 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1514.g25296a8f 2024-02-26T05:57:59.986 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-26T05:57:59.992 INFO:teuthology.task.internal:no buildpackages task found 2024-02-26T05:57:59.992 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-26T05:58:00.028 INFO:teuthology.task.internal:Saving configuration 2024-02-26T05:58:00.054 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-26T05:58:00.095 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T05:58:00.119 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:52:17.533602', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZPdkDZSRXwvoUvPcwAYAldqb0CHJhFpgKlOxRkyHNF6rbw1E2/xgFSq4EqU7IaDXqrmGdkezUdpgdkbmhtm9I='} 2024-02-26T05:58:00.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:52:17.532513', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCWZElmQG/kXQ7G++W9esi9RidN6ruebCJTuqMKCH6GTXUldq95uZ52llHODIcBZYdrfxfYQpcyH1IpskP5OiY='} 2024-02-26T05:58:00.164 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:52:17.531197', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJN6FQPyU3rQNmlN7XjX9ozDAbCOMd7C4QTI8TsxNvfiK+XiywOGT8wepALDNcH6c3vpDYA/5J2Lik6ezL6wxo='} 2024-02-26T05:58:00.164 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-26T05:58:00.176 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-26T05:58:00.176 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-26T05:58:00.176 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-26T05:58:00.177 INFO:teuthology.run_tasks:Running task console_log... 2024-02-26T05:58:00.292 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7509e1c0d0>, signals=[15]) 2024-02-26T05:58:00.293 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-26T05:58:00.298 INFO:teuthology.task.internal:Opening connections... 2024-02-26T05:58:00.299 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T05:58:00.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:58:00.414 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-02-26T05:58:00.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:58:00.523 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-02-26T05:58:00.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T05:58:00.673 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-26T05:58:00.698 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-02-26T05:58:00.720 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-02-26T05:58:00.721 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-02-26T05:58:00.782 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T05:58:00.783 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T05:58:00.784 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-02-26T05:58:00.808 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-02-26T05:58:00.830 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-02-26T05:58:00.830 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-02-26T05:58:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-02-26T05:58:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-02-26T05:58:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-02-26T05:58:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-02-26T05:58:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-02-26T05:58:00.887 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T05:58:00.888 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T05:58:00.888 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-02-26T05:58:00.908 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-02-26T05:58:00.926 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-02-26T05:58:00.926 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-02-26T05:58:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T05:58:00.985 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T05:58:00.985 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T05:58:00.985 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-02-26T05:58:01.008 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-26T05:58:01.018 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-26T05:58:01.045 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-26T05:58:01.045 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-02-26T05:58:01.048 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-02-26T05:58:01.050 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-02-26T05:58:01.067 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-26T05:58:01.071 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-26T05:58:01.072 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-02-26T05:58:01.105 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-02-26T05:58:01.108 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-02-26T05:58:01.144 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-26T05:58:01.233 INFO:teuthology.run_tasks:Running task kernel... 2024-02-26T05:58:01.251 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '7c0fbba6c508351b48a34af7de7b074eb29ad0f9'} 2024-02-26T05:58:01.251 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-02-26T05:58:01.251 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-02-26T05:58:01.251 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T05:58:01.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:58:01.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-26T05:58:01.627 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T05:58:01.627 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7c0fbba6c508351b48a34af7de7b074eb29ad0f9"... 2024-02-26T05:58:01.627 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-02-26T05:58:01.645 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-361.el9.x86_64 2024-02-26T05:58:01.645 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T05:58:01.646 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-26T05:58:01.646 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum upgrade -y linux-firmware 2024-02-26T05:58:03.943 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-02-26T05:58:05.694 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.5 MB 00:00 2024-02-26T05:58:07.859 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-02-26T05:58:11.087 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 19 MB 00:02 2024-02-26T05:58:16.419 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.1 MB 00:01 2024-02-26T05:58:18.053 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-02-26T05:58:19.773 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-26T05:58:24.979 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr:Modular dependency problems: 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 1: conflicting requests 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 2: conflicting requests 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 3: conflicting requests 2024-02-26T05:58:27.345 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 4: conflicting requests 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 5: conflicting requests 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 6: conflicting requests 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 7: conflicting requests 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 8: conflicting requests 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 9: conflicting requests 2024-02-26T05:58:27.346 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 10: conflicting requests 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 11: conflicting requests 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 12: conflicting requests 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 13: conflicting requests 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.347 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 14: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 15: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 16: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 17: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 18: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 19: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 20: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 21: conflicting requests 2024-02-26T05:58:27.348 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 22: conflicting requests 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 23: conflicting requests 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 24: conflicting requests 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 25: conflicting requests 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 26: conflicting requests 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 27: conflicting requests 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.349 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 28: conflicting requests 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 29: conflicting requests 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 30: conflicting requests 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 31: conflicting requests 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 32: conflicting requests 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 33: conflicting requests 2024-02-26T05:58:27.350 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 34: conflicting requests 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 35: conflicting requests 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 36: conflicting requests 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T05:58:27.351 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 38: conflicting requests 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.352 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.353 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.354 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T05:58:27.884 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T05:58:27.902 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-02-26T05:58:27.903 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-02-26T05:58:27.903 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 1 Package 2024-02-26T05:58:27.903 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T05:58:27.903 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 354 M 2024-02-26T05:58:27.903 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-02-26T05:58:37.647 INFO:teuthology.orchestra.run.smithi006.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 36 MB/s | 354 MB 00:09 2024-02-26T05:58:37.648 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-02-26T05:58:37.649 INFO:teuthology.orchestra.run.smithi006.stdout:Total 36 MB/s | 354 MB 00:09 2024-02-26T05:58:37.649 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-02-26T05:58:37.742 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-02-26T05:58:37.742 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-02-26T05:58:39.336 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-02-26T05:58:39.336 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-02-26T05:59:01.867 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-02-26T05:59:02.149 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T05:59:02.262 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T05:59:09.713 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T05:59:09.713 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T05:59:10.130 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T05:59:10.130 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T05:59:10.130 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-02-26T05:59:10.130 INFO:teuthology.orchestra.run.smithi006.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-02-26T05:59:10.131 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T05:59:10.131 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-02-26T05:59:10.428 INFO:teuthology.task.kernel:Downloading kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T05:59:10.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T05:59:10.578 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ 2024-02-26T05:59:10.579 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-26T05:59:10.581 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi006.front.sepia.ceph.com to complete... 2024-02-26T05:59:32.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26 05:59:32 URL:https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm [335934793/335934793] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-26T05:59:32.572 INFO:teuthology.orchestra.run.smithi006.stderr:FINISHED --2024-02-26 05:59:32-- 2024-02-26T05:59:32.572 INFO:teuthology.orchestra.run.smithi006.stderr:Total wall clock time: 22s 2024-02-26T05:59:32.572 INFO:teuthology.orchestra.run.smithi006.stderr:Downloaded: 1 files, 320M in 22s (14.8 MB/s) 2024-02-26T05:59:32.575 INFO:teuthology.task.kernel:Installing kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T05:59:32.575 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-26T05:59:34.599 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying... ######################################## 2024-02-26T05:59:34.677 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing... ######################################## 2024-02-26T05:59:34.761 INFO:teuthology.orchestra.run.smithi006.stdout:Updating / installing... 2024-02-26T05:59:42.600 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-6.8.0_rc4_g7c0fbba6c508-1 ######################################## 2024-02-26T05:59:58.271 INFO:teuthology.orchestra.run.smithi006.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-26T06:00:21.606 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-26T06:00:21.606 DEBUG:teuthology.orchestra.run.smithi006:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-26T06:00:21.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:00:21.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/System.map 2024-02-26T06:00:21.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/config 2024-02-26T06:00:21.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel 2024-02-26T06:00:21.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch 2024-02-26T06:00:21.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-26T06:00:21.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-26T06:00:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ansi_cprng.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/anubis.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/arc4.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_memcpy.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_pq.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_tx.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_xor.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/raid6test.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authenc.ko 2024-02-26T06:00:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authencesn.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blake2b_generic.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_common.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_generic.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/camellia_generic.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast5_generic.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast6_generic.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast_common.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ccm.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cmac.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crc32_generic.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_engine.ko 2024-02-26T06:00:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_user.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cts.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/des_generic.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/dh_generic.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/echainiv.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/essiv.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/fcrypt.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/khazad.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4hc.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/md4.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/michael_mic.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcbc.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcrypt.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/rmd160.ko 2024-02-26T06:00:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/seed.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/serpent_generic.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tcrypt.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tea.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_common.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_generic.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/vmac.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/wp512.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xcbc.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xor.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xxhash_generic.ko 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi 2024-02-26T06:00:21.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_pad.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/custom_method.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/ec_sys.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/platform_profile.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbs.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbshc.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/video.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/acard-ahci.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ahci_platform.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ata_generic.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/libahci_platform.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_acpi.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ali.ko 2024-02-26T06:00:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_amd.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_artop.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atiixp.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atp867x.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd640.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd64x.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cypress.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_efar.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt366.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt37x.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it8213.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it821x.ko 2024-02-26T06:00:21.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_jmicron.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_marvell.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_mpiix.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_netcell.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ninja32.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87410.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87415.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_oldpiix.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_opti.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_optidma.ko 2024-02-26T06:00:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pcmcia.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_piccolo.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_rdc.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sch.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_serverworks.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sil680.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sis.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sl82c105.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_triflex.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_via.ko 2024-02-26T06:00:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pdc_adma.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_inic162x.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_mv.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_nv.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_promise.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_qstor.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil24.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sis.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_svw.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sx4.ko 2024-02-26T06:00:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_uli.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_via.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_vsc.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay/ks0108.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma/bcma.ko 2024-02-26T06:00:21.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe/aoe.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/brd.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd/drbd.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/floppy.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/loop.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/nbd.ko 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk 2024-02-26T06:00:21.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk/null_blk.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/pktcdvd.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/rbd.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/virtio_blk.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkfront.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram/zram.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hangcheck-timer.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-26T06:00:21.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/via-rng.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/lp.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave/mwave.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ppdev.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tlclk.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm.ko 2024-02-26T06:00:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/uv_mmtimer.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/virtio_console.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-26T06:00:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-aes.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-sha.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca 2024-02-26T06:00:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca/dca.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/amd64_edac.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/e752x_edac.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/edac_mce_amd.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3000_edac.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3200_edac.ko 2024-02-26T06:00:21.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5100_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5400_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7300_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7core_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i82975x_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/ie31200_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/sb_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/x38_edac.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/edd.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio 2024-02-26T06:00:21.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-ich.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-appleir.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-aureal.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-axff.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-dr.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elecom.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elo.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-emsff.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gaff.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gt683r.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gyration.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-26T06:00:21.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtekff.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-hyperv.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-icade.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-keytouch.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-kye.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lcpower.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-led.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lenovo.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-26T06:00:21.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-multitouch.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-ortek.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-penmount.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-petalynx.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-picolcd.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-pl.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-plantronics.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-primax.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-rmi.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-common.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-26T06:00:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-saitek.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-samsung.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sjoy.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sony.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-speedlink.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-steelseries.ko 2024-02-26T06:00:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sunplus.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tivo.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tmff.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-topseed.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-twinhan.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-uclogic.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-waltop.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-wiimote.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-xinmo.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zpff.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zydacron.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/uhid.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/wacom.ko 2024-02-26T06:00:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_balloon.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_utils.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_vmbus.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru3.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7414.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7418.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adc128d818.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1025.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1026.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1029.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1031.ko 2024-02-26T06:00:21.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm9240.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ads7828.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7410.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7411.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7462.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7470.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7475.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7x10.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/amc6821.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/applesmc.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asb100.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asc7621.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-26T06:00:21.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/atxp1.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/coretemp.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dme1737.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds1621.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds620.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc1403.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc6w201.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71805f.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71882fg.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f75375s.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fam15h_power.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fschmd.ko 2024-02-26T06:00:21.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g760a.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g762.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl518sm.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl520sm.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5500_temp.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5k_amb.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmaem.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmpex.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina209.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina2xx.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/it87.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k10temp.ko 2024-02-26T06:00:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k8temp.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lineage-pem.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm63.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm73.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm75.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm77.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm78.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm80.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm83.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm85.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm87.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm90.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm92.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm93.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95234.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95241.ko 2024-02-26T06:00:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95245.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc2945.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4151.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4215.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4222.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4245.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4260.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4261.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max16065.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1619.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1668.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max197.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6639.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6650.ko 2024-02-26T06:00:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6697.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/mcp3021.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6683.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775-core.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct7802.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87360.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87427.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pcf8591.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-26T06:00:21.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/powr1220.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5627.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5636.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht15.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht21.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/shtc1.ko 2024-02-26T06:00:21.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sis5595.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47b397.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m1.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m192.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/thmc50.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp102.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp103.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp401.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp421.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via-cputemp.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via686a.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt1211.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt8231.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627ehf.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627hf.ko 2024-02-26T06:00:21.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83781d.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83791d.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83792d.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83793.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83795.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l785ts.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l786ng.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-26T06:00:21.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-26T06:00:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-dev.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-mux.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-smbus.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-stub.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-26T06:00:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_core.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-26T06:00:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-26T06:00:21.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/gameport.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/joydev.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/appletouch.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/bcm5974.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/cyapatp.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/sermouse.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-26T06:00:21.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/altera_ps2.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/arc_ps2.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/serio_raw.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/sparse-keymap.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-blinkm.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lm3530.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lp3944.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-ss4200.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-26T06:00:21.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache/bcache.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-bio-prison.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache-smq.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-crypt.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-delay.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-flakey.ko 2024-02-26T06:00:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-log-userspace.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-multipath.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-queue-length.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-raid.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-round-robin.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-service-time.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-switch.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-thin-pool.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-verity.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid0.ko 2024-02-26T06:00:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid1.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid10.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid456.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptbase.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptctl.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptfc.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptlan.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptsas.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptscsih.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptspi.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_ich.ko 2024-02-26T06:00:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_sch.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/mfd-core.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/sm501.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/viperboard.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/vx855.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/wl1273-core.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds9802als.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds990x.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/bh1770glc.ko 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710 2024-02-26T06:00:21.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710/cb710.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/at24.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/max6875.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/enclosure.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/hpilo.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29003.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29020.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-me.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-txe.ko 2024-02-26T06:00:21.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_7xx1.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_core.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tsl2550.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_balloon.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net 2024-02-26T06:00:21.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding/bonding.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dummy.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/eql.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-26T06:00:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera 2024-02-26T06:00:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-26T06:00:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio 2024-02-26T06:00:21.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-26T06:00:21.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dnet.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ethoc.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/fealnx.ko 2024-02-26T06:00:21.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf 2024-02-26T06:00:21.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/jme.ko 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell 2024-02-26T06:00:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-26T06:00:21.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-26T06:00:21.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc 2024-02-26T06:00:21.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-26T06:00:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet 2024-02-26T06:00:21.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/geneve.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ifb.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-26T06:00:21.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvlan.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvtap.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mii.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/net_failover.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/netconsole.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/nlmon.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy 2024-02-26T06:00:21.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/amd.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm87xx.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/broadcom.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/cicada.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/davicom.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/dp83640.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/et1011c.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/icplus.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/lxt.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/marvell.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/micrel.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/national.ko 2024-02-26T06:00:21.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/phylink.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/qsemi.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/realtek.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/smsc.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/ste10Xp.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/vitesse.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_async.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppoe.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppox.ko 2024-02-26T06:00:21.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pptp.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slhc.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slip.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/sungem_phy.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tap.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_random.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tun.ko 2024-02-26T06:00:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/veth.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/virtio_net.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan/vxlan.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netfront.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb/ntb.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme-core.ko 2024-02-26T06:00:21.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme.ko 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport.ko 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_cs.ko 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_pc.ko 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_serial.ko 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie 2024-02-26T06:00:21.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/xen-pcifront.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/i82092.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/pd6729.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-26T06:00:21.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acerhdf.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/hdaps.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-26T06:00:21.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel_ips.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps 2024-02-26T06:00:21.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps_parport.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/pps_core.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp_kvm.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-26T06:00:21.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-em3027.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-max6900.ko 2024-02-26T06:00:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-x1205.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-9xxx.ko 2024-02-26T06:00:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-sas.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-xxxx.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/BusLogic.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/a100u2w.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/advansys.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/am53c974.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-26T06:00:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/atp870u.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa/bfa.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ch.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-26T06:00:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dc395x.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dmx3191d.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esp_scsi.ko 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe 2024-02-26T06:00:21.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic/fnic.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hpsa.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hptiop.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hv_storvsc.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/initio.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ipr.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ips.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci/isci.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc 2024-02-26T06:00:21.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc/libfc.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas/libsas.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-26T06:00:21.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvumi.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pmcraid.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla1280.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/raid_class.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_debug.ko 2024-02-26T06:00:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ses.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/st.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/stex.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/virtio_scsi.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/wd719x.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb/ssb.ko 2024-02-26T06:00:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_file.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_iblock.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_mod.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_pscsi.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_user.ko 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc 2024-02-26T06:00:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-26T06:00:21.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless 2024-02-26T06:00:21.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_gsm.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_hdlc.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/nozomi.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/arc_uart.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host 2024-02-26T06:00:21.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_aec.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_cif.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_pci_generic.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_sercos3.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-acm.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usblp.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usbtmc.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-26T06:00:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/mdc800.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/microtek.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/adutux.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/appledisplay.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi26.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi62.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ezusb.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/idmouse.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/iowarrior.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ldusb.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/legousbtower.ko 2024-02-26T06:00:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usb3503.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usblcd.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/uss720.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/yurex.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy/phy-generic.ko 2024-02-26T06:00:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/aircable.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ark3116.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ch341.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cp210x.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cyberjack.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/empeg.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-26T06:00:21.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_ti.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipaq.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipw.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ir-usb.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mct_u232.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7720.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7840.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/navman.ko 2024-02-26T06:00:21.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/omninet.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/opticon.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/option.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/oti6858.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/pl2303.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcaux.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcserial.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/quatech2.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/safe_serial.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/sierra.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ssu100.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/symbolserial.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-26T06:00:21.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_debug.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/visor.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/whiteheat.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/uas.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-karma.ko 2024-02-26T06:00:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/usb-storage.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio.ko 2024-02-26T06:00:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_net.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_scsi.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/apple_bl.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/lcd.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_balloon.ko 2024-02-26T06:00:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_mmio.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_ring.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2482.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2490.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-26T06:00:21.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/wire.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/hpwdt.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/i6300esb.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ib700wdt.ko 2024-02-26T06:00:21.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ibmasr.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it87_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/machzwd.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/nv_tco.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/softdog.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/via_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-26T06:00:21.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/wdt_pci.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/xen_wdt.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-evtchn.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntalloc.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntdev.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-privcmd.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p 2024-02-26T06:00:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p/9p.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs/affs.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs/befs.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/binfmt_misc.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs/btrfs.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles/cachefiles.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph/ceph.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda/coda.ko 2024-02-26T06:00:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs/cramfs.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm/dlm.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs/f2fs.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/fat.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/msdos.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/vfat.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/cuse.ko 2024-02-26T06:00:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/fuse.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2/gfs2.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs/hfs.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus/hfsplus.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs/isofs.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs/jfs.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd/lockd.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix/minix.ko 2024-02-26T06:00:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs/netfs.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfs.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv3.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv4.ko 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common 2024-02-26T06:00:21.784 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/grace.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/nfs_acl.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd/nfsd.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2/nilfs2.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-celtic.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-centeuro.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-croatian.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-cyrillic.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-gaelic.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-greek.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-iceland.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-inuit.ko 2024-02-26T06:00:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-roman.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-romanian.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-turkish.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1250.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1251.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1255.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp737.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp775.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp850.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp852.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp855.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp857.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp860.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp861.ko 2024-02-26T06:00:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp862.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp863.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp864.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp865.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp866.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp869.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp874.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp932.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp936.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp949.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp950.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_euc-jp.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-1.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-13.ko 2024-02-26T06:00:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-14.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-15.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-2.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-3.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-4.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-5.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-6.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-7.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-9.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-r.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-ru.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-u.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_utf8.ko 2024-02-26T06:00:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs/overlay.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore/ramoops.ko 2024-02-26T06:00:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs/reiserfs.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs/romfs.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client/cifs.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_arc4.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_md4.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs/squashfs.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv/sysv.ko 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf 2024-02-26T06:00:21.790 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf/udf.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs/ufs.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs/xfs.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu/rcutorture.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/torture.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/asn1_encoder.ko 2024-02-26T06:00:21.791 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc-itu-t.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc8.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libarc4.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libdes.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/libcrc32c.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lru_cache.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4_compress.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4hc_compress.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math/cordic.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6/raid6_pq.ko 2024-02-26T06:00:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_bm.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_fsm.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_kmp.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd/zstd_compress.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm/hwpoison-inject.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/6lowpan.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_dest.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_fragment.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_hop.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-26T06:00:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_mobility.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_routing.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_udp.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/garp.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/mrp.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/p8022.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/psnap.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/stp.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q/8021q.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_fd.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_rdma.ko 2024-02-26T06:00:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_virtio.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv/batman-adv.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/br_netfilter.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/bridge.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-26T06:00:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtables.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-26T06:00:21.796 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph/libceph.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/failover.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/pktgen.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_diag.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv4.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv6.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver/dns_resolver.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/dsa_core.ko 2024-02-26T06:00:21.797 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_brcm.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_dsa.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_none.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_trailer.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ah4.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/esp4.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/fou.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/gre.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/inet_diag.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_gre.ko 2024-02-26T06:00:21.798 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_tunnel.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_vti.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipcomp.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipip.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-26T06:00:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_bic.ko 2024-02-26T06:00:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_dctcp.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_diag.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_highspeed.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_htcp.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_hybla.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_illinois.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_lp.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_scalable.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_vegas.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_veno.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_westwood.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_yeah.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tunnel4.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_diag.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_tunnel.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-26T06:00:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ah6.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/esp6.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/fou6.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_tunnel.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_vti.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ipcomp6.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-26T06:00:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-26T06:00:21.803 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/sit.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/tunnel6.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key/af_key.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_core.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_eth.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip.ko 2024-02-26T06:00:21.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip6.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_netlink.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ppp.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc/llc.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154/mac802154.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls/mpls_gso.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-26T06:00:21.805 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-26T06:00:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-26T06:00:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conncount.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-26T06:00:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_log_syslog.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_irc.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_sip.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-26T06:00:21.809 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_tables.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_log.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_chain_nat.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_compat.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_ct.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fib.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_hash.ko 2024-02-26T06:00:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_limit.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_log.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_masq.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_nat.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_numgen.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_queue.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_quota.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_redir.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject_inet.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_socket.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_tunnel.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_AUDIT.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-26T06:00:21.811 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CT.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_DSCP.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HL.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HMARK.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LED.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LOG.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NETMAP.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFLOG.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_RATEEST.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_SECMARK.ko 2024-02-26T06:00:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TEE.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TPROXY.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TRACE.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_addrtype.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_bpf.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cgroup.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cluster.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_comment.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connbytes.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlabel.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlimit.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connmark.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_conntrack.ko 2024-02-26T06:00:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cpu.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dccp.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_devgroup.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dscp.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ecn.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_esp.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hashlimit.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_helper.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hl.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipcomp.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_iprange.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipvs.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_l2tp.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_length.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_limit.ko 2024-02-26T06:00:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mac.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mark.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_multiport.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nat.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nfacct.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_osf.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_owner.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_physdev.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_pkttype.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_policy.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_quota.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_rateest.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_realm.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_recent.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_sctp.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_set.ko 2024-02-26T06:00:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_socket.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_state.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_statistic.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_string.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_tcpmss.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_time.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_u32.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink/netlink_diag.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh/nsh.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/openvswitch.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-geneve.ko 2024-02-26T06:00:21.816 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-gre.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-vxlan.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet/af_packet_diag.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_rdma.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_tcp.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_csum.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_gact.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_mirred.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_nat.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_pedit.ko 2024-02-26T06:00:21.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_police.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_simple.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_skbedit.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_vlan.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_basic.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_bpf.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_flow.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_fw.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_route.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_u32.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_cmp.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_ipset.ko 2024-02-26T06:00:21.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_meta.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_nbyte.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_text.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_u32.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_choke.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_codel.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_drr.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_fq.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_gred.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hfsc.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hhf.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_htb.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_ingress.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio_lib.ko 2024-02-26T06:00:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_multiq.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_netem.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_pie.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_plug.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_prio.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_qfq.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_red.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfb.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfq.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_tbf.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_teql.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp_diag.ko 2024-02-26T06:00:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/sunrpc.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/diag.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/tipc.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix/unix_diag.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-26T06:00:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys/trusted.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib/irqbypass.ko 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias 2024-02-26T06:00:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias.bin 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.alias.bin 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.bin 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.modinfo 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep.bin 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.devname 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.order 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.softdep 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols.bin 2024-02-26T06:00:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T06:00:21.827 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:00:21.828 DEBUG:teuthology.orchestra.run.smithi006:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-26T06:00:21.850 INFO:teuthology.orchestra.run.smithi006.stdout:postinstall scriptlet (using /bin/sh): 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: /usr/bin/kernel-install add 6.8.0-rc4-g7c0fbba6c508 /lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:for file in vmlinuz System.map config; do 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508"; then 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: cp "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508" 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: fi 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:done 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:new-kernel-pkg --remove 6.8.0-rc4-g7c0fbba6c508 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc4-g7c0fbba6c508.img 2024-02-26T06:00:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-26T06:00:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-install remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:00:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-02-26T06:00:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-26T06:00:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-26T06:00:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/update-bootloader --remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:00:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-02-26T06:00:21.853 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2024-02-26T06:00:21.934 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 68.el9 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8264254 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-26T06:00:21.935 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:00:21.936 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2024-02-26T06:00:21.937 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-26T06:00:21.937 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:00:21.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-26T06:00:22.562 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-02-26T06:00:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-26T06:00:23.990 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-02-26T06:00:23.992 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-02-26T06:00:24.009 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.ZqFLSptdk5 2024-02-26T06:00:24.009 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZqFLSptdk5 2024-02-26T06:00:24.081 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.ZqFLSptdk5 2024-02-26T06:00:24.230 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.ZqFLSptdk5 is 6KB 2024-02-26T06:00:24.278 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.ZqFLSptdk5 2024-02-26T06:00:24.293 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2024-02-26T06:00:24.361 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-26T06:00:24.362 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-26T06:00:24.362 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-26T06:00:24.362 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508.conf 2024-02-26T06:00:24.363 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:00:24.507 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-02-26T06:00:54.511 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-26T06:00:54.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-02-26T06:00:54.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:01:54.515 DEBUG:teuthology.orchestra.remote:timed out 2024-02-26T06:02:03.516 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-02-26T06:02:03.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:02:04.025 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-02-26T06:02:04.723 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-02-26T06:02:04.723 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc4-g7c0fbba6c508"... 2024-02-26T06:02:04.723 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-02-26T06:02:04.785 INFO:teuthology.orchestra.run.smithi006.stdout:6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:02:04.785 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc4-g7c0fbba6c508 vs 6.8.0-rc4-g7c0fbba6c508 2024-02-26T06:02:04.785 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-26T06:02:04.785 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-26T06:02:05.788 DEBUG:teuthology.parallel:result is None 2024-02-26T06:02:05.789 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-26T06:02:05.796 INFO:teuthology.task.internal:Creating test directory... 2024-02-26T06:02:05.796 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T06:02:05.801 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T06:02:05.804 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T06:02:05.840 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-26T06:02:05.847 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-26T06:02:05.870 INFO:teuthology.task.internal:Creating archive directory... 2024-02-26T06:02:05.870 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T06:02:05.886 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T06:02:05.889 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T06:02:05.952 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-26T06:02:05.958 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-26T06:02:05.958 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T06:02:05.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:02:05.977 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T06:02:05.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:02:05.994 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T06:02:06.000 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T06:02:06.025 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T06:02:06.076 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T06:02:06.085 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T06:02:06.098 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T06:02:06.107 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T06:02:06.138 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T06:02:06.190 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T06:02:06.194 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-26T06:02:06.202 INFO:teuthology.task.internal:Configuring sudo... 2024-02-26T06:02:06.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T06:02:06.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T06:02:06.220 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T06:02:06.287 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-26T06:02:06.297 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-26T06:02:06.297 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T06:02:06.303 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T06:02:06.306 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T06:02:06.335 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T06:02:06.454 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T06:02:06.567 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T06:02:06.636 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T06:02:06.739 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:02:06.740 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T06:02:06.874 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T06:02:06.918 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T06:02:06.971 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T06:02:07.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T06:02:07.105 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:02:07.106 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T06:02:07.173 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T06:02:07.214 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T06:02:07.279 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T06:02:07.358 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T06:02:07.409 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:02:07.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T06:02:07.481 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-02-26T06:02:07.487 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-02-26T06:02:07.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-02-26T06:02:07.520 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:02:07.554 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:02:07.589 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:02:08.019 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-26T06:02:08.028 INFO:teuthology.task.internal:Starting timer... 2024-02-26T06:02:08.028 INFO:teuthology.run_tasks:Running task pcp... 2024-02-26T06:02:08.159 INFO:teuthology.run_tasks:Running task selinux... 2024-02-26T06:02:08.239 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-26T06:02:08.240 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2024-02-26T06:02:08.368 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2024-02-26T06:02:08.372 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-02-26T06:02:08.442 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-02-26T06:02:08.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-02-26T06:02:08.516 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-02-26T06:02:08.518 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-26T06:02:08.519 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2024-02-26T06:02:08.589 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2024-02-26T06:02:08.590 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-02-26T06:02:08.610 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-02-26T06:02:08.611 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-02-26T06:02:08.636 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-02-26T06:02:08.636 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-02-26T06:02:08.636 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:02:08.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:02:08.717 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:02:08.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:02:08.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:02:08.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:02:08.774 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-26T06:02:08.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2024-02-26T06:02:08.859 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-02-26T06:02:08.894 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-02-26T06:02:08.929 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-26T06:02:08.942 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-26T06:02:08.982 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-26T06:02:08.984 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi090.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-26T06:08:56.541 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-02-26T06:08:56.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-02-26T06:08:56.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:08:56.719 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-02-26T06:08:56.877 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-02-26T06:08:56.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-02-26T06:08:56.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:08:56.967 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-02-26T06:08:57.036 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-02-26T06:08:57.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-02-26T06:08:57.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T06:08:57.131 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-02-26T06:08:57.200 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-02-26T06:08:57.200 INFO:teuthology.run_tasks:Running task clock... 2024-02-26T06:08:57.210 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-26T06:08:57.211 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T06:08:57.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:08:57.218 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T06:08:57.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:08:57.221 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T06:08:57.221 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:08:57.253 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T06:08:57.271 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T06:08:57.275 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T06:08:57.294 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T06:08:57.301 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-02-26T06:08:57.315 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-02-26T06:08:57.322 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T06:08:57.325 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-02-26T06:08:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T06:08:57.339 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-02-26T06:08:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T06:08:57.356 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T06:08:57.373 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T06:08:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T06:08:57.392 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:08:57.395 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:08:57.395 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-02-26T06:08:57.395 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.395 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.395 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.395 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.417 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:08:57.420 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:08:57.420 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-02-26T06:08:57.420 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.420 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.420 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.420 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2024-02-26T06:08:57.525 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2024-02-26T06:08:57.581 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T06:08:57.638 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T06:08:57.766 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:08:57.774 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:08:57.774 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-02-26T06:08:57.774 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.774 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.774 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.774 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:08:57.776 INFO:teuthology.run_tasks:Running task install... 2024-02-26T06:08:57.784 DEBUG:teuthology.task.install:project ceph 2024-02-26T06:08:57.784 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33'}} 2024-02-26T06:08:57.784 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33'} 2024-02-26T06:08:57.784 INFO:teuthology.task.install:Using flavor: default 2024-02-26T06:08:57.790 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-26T06:08:57.790 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-26T06:08:57.791 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T06:08:57.791 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:08:57.793 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T06:08:57.794 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:08:57.794 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T06:08:57.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:08:57.956 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T06:08:57.956 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T06:08:58.039 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T06:08:58.040 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T06:08:58.045 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T06:08:58.045 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T06:08:58.229 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T06:08:58.230 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:08:58.230 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T06:08:58.260 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T06:08:58.261 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T06:08:58.307 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T06:08:58.308 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:08:58.308 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T06:08:58.314 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T06:08:58.314 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:08:58.314 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T06:08:58.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T06:08:58.345 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T06:08:58.345 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T06:08:58.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T06:08:58.409 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T06:08:58.413 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T06:08:58.414 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T06:08:58.481 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-02-26T06:08:58.482 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-02-26T06:08:58.496 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T06:08:58.553 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T06:08:58.564 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-02-26T06:08:58.565 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-02-26T06:08:58.687 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T06:08:58.719 INFO:teuthology.orchestra.run.smithi090.stdout:74 files removed 2024-02-26T06:08:58.746 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T06:08:58.774 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2024-02-26T06:08:58.778 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2024-02-26T06:08:58.802 INFO:teuthology.orchestra.run.smithi167.stdout:74 files removed 2024-02-26T06:08:58.829 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T06:08:59.152 INFO:teuthology.orchestra.run.smithi006.stdout:74 files removed 2024-02-26T06:08:59.193 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T06:08:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 102 kB/s | 82 kB 00:00 2024-02-26T06:09:00.202 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 76 kB/s | 82 kB 00:01 2024-02-26T06:09:00.555 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-02-26T06:09:00.558 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 98 kB/s | 82 kB 00:00 2024-02-26T06:09:00.902 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-02-26T06:09:01.315 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-02-26T06:09:01.425 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-02-26T06:09:01.711 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-02-26T06:09:01.842 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - AppStream 16 MB/s | 8.1 MB 00:00 2024-02-26T06:09:02.201 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-02-26T06:09:02.214 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS-9 - AppStream 17 MB/s | 8.1 MB 00:00 2024-02-26T06:09:02.963 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-02-26T06:09:03.568 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.5 MB 00:00 2024-02-26T06:09:03.873 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.5 MB 00:00 2024-02-26T06:09:04.964 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.5 MB 00:00 2024-02-26T06:09:06.038 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-02-26T06:09:06.757 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-02-26T06:09:08.481 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-02-26T06:09:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.1 MB 00:01 2024-02-26T06:09:14.467 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 781 kB/s | 8.0 MB 00:10 2024-02-26T06:09:14.858 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2024-02-26T06:09:17.728 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-02-26T06:09:22.038 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.1 MB 00:00 2024-02-26T06:09:23.514 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 15 kB 00:00 2024-02-26T06:09:24.532 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2024-02-26T06:09:29.542 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-02-26T06:09:30.391 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 876 kB/s | 19 MB 00:22 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr:Modular dependency problems: 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 1: conflicting requests 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 2: conflicting requests 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.130 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 3: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 4: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 5: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 6: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 7: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 8: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 9: conflicting requests 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.131 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 10: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 11: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 12: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 13: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 14: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 15: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 16: conflicting requests 2024-02-26T06:09:32.132 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 17: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 18: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 19: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 20: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 21: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 22: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 23: conflicting requests 2024-02-26T06:09:32.133 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 24: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 25: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 26: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 27: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 28: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 29: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 30: conflicting requests 2024-02-26T06:09:32.134 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 31: conflicting requests 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 32: conflicting requests 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 33: conflicting requests 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 34: conflicting requests 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 35: conflicting requests 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 36: conflicting requests 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.135 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 38: conflicting requests 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T06:09:32.136 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.137 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.138 INFO:teuthology.orchestra.run.smithi090.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T06:09:32.538 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:09:32.539 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:09:32.551 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T06:09:32.552 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T06:09:32.553 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T06:09:32.554 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T06:09:32.556 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T06:09:32.559 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T06:09:32.560 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T06:09:32.647 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout:========================================================================================= 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout:========================================================================================= 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T06:09:32.672 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T06:09:32.673 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T06:09:32.674 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T06:09:32.675 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T06:09:32.676 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T06:09:32.677 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T06:09:32.678 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T06:09:32.679 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T06:09:32.680 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T06:09:32.681 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T06:09:32.682 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T06:09:32.683 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T06:09:32.684 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T06:09:32.685 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-02-26T06:09:32.686 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T06:09:32.687 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:09:32.687 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-02-26T06:09:32.687 INFO:teuthology.orchestra.run.smithi090.stdout:========================================================================================= 2024-02-26T06:09:32.687 INFO:teuthology.orchestra.run.smithi090.stdout:Install 150 Packages 2024-02-26T06:09:32.687 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 16 Packages 2024-02-26T06:09:32.687 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:09:32.688 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 221 M 2024-02-26T06:09:32.688 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-02-26T06:09:33.818 INFO:teuthology.orchestra.run.smithi090.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-02-26T06:09:34.745 INFO:teuthology.orchestra.run.smithi090.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 956 kB/s | 886 kB 00:00 2024-02-26T06:09:34.937 INFO:teuthology.orchestra.run.smithi090.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 758 kB/s | 145 kB 00:00 2024-02-26T06:09:36.254 INFO:teuthology.orchestra.run.smithi090.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 1.6 MB/s | 2.1 MB 00:01 2024-02-26T06:09:36.377 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.1 MB 00:01 2024-02-26T06:09:36.488 INFO:teuthology.orchestra.run.smithi090.stdout:(5/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 1.8 MB/s | 5.3 MB 00:03 2024-02-26T06:09:36.830 INFO:teuthology.orchestra.run.smithi090.stdout:(6/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-02-26T06:09:37.522 INFO:teuthology.orchestra.run.smithi090.stdout:(7/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-02-26T06:09:39.252 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 15 kB 00:01 2024-02-26T06:09:40.005 INFO:teuthology.orchestra.run.smithi090.stdout:(8/166): ceph-radosgw-19.0.0-1514.g25296a8f.el9 3.9 MB/s | 9.7 MB 00:02 2024-02-26T06:09:40.097 INFO:teuthology.orchestra.run.smithi090.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 275 kB/s | 25 kB 00:00 2024-02-26T06:09:40.954 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2024-02-26T06:09:41.064 INFO:teuthology.orchestra.run.smithi090.stdout:(10/166): ceph-common-19.0.0-1514.g25296a8f.el9 2.6 MB/s | 20 MB 00:07 2024-02-26T06:09:41.281 INFO:teuthology.orchestra.run.smithi090.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 1.0 MB/s | 217 kB 00:00 2024-02-26T06:09:41.389 INFO:teuthology.orchestra.run.smithi090.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 250 kB/s | 27 kB 00:00 2024-02-26T06:09:41.490 INFO:teuthology.orchestra.run.smithi090.stdout:(13/166): libcephfs-devel-19.0.0-1514.g25296a8f 312 kB/s | 31 kB 00:00 2024-02-26T06:09:41.990 INFO:teuthology.orchestra.run.smithi090.stdout:(14/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 1.5 MB/s | 743 kB 00:00 2024-02-26T06:09:42.099 INFO:teuthology.orchestra.run.smithi090.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.5 MB/s | 163 kB 00:00 2024-02-26T06:09:42.191 INFO:teuthology.orchestra.run.smithi090.stdout:(16/166): librados-devel-19.0.0-1514.g25296a8f. 1.4 MB/s | 127 kB 00:00 2024-02-26T06:09:42.475 INFO:teuthology.orchestra.run.smithi090.stdout:(17/166): libradosstriper1-19.0.0-1514.g25296a8 1.7 MB/s | 498 kB 00:00 2024-02-26T06:09:43.000 INFO:teuthology.orchestra.run.smithi090.stdout:(18/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x8 2.7 MB/s | 17 MB 00:06 2024-02-26T06:09:43.092 INFO:teuthology.orchestra.run.smithi090.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 493 kB/s | 45 kB 00:00 2024-02-26T06:09:43.184 INFO:teuthology.orchestra.run.smithi090.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 1.4 MB/s | 131 kB 00:00 2024-02-26T06:09:43.301 INFO:teuthology.orchestra.run.smithi090.stdout:(21/166): python3-cephfs-19.0.0-1514.g25296a8f. 1.4 MB/s | 161 kB 00:00 2024-02-26T06:09:43.485 INFO:teuthology.orchestra.run.smithi090.stdout:(22/166): python3-rados-19.0.0-1514.g25296a8f.e 1.7 MB/s | 320 kB 00:00 2024-02-26T06:09:43.660 INFO:teuthology.orchestra.run.smithi090.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 1.7 MB/s | 298 kB 00:00 2024-02-26T06:09:43.777 INFO:teuthology.orchestra.run.smithi090.stdout:(24/166): python3-rgw-19.0.0-1514.g25296a8f.el9 852 kB/s | 99 kB 00:00 2024-02-26T06:09:43.894 INFO:teuthology.orchestra.run.smithi090.stdout:(25/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 731 kB/s | 85 kB 00:00 2024-02-26T06:09:45.429 INFO:teuthology.orchestra.run.smithi090.stdout:(26/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 2.0 MB/s | 3.1 MB 00:01 2024-02-26T06:09:45.537 INFO:teuthology.orchestra.run.smithi090.stdout:(27/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 1.5 MB/s | 171 kB 00:00 2024-02-26T06:09:45.646 INFO:teuthology.orchestra.run.smithi090.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 290 kB/s | 31 kB 00:00 2024-02-26T06:09:45.747 INFO:teuthology.orchestra.run.smithi090.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 1.3 MB/s | 138 kB 00:00 2024-02-26T06:09:45.963 INFO:teuthology.orchestra.run.smithi090.stdout:(30/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 1.5 MB/s | 5.1 MB 00:03 2024-02-26T06:09:46.614 INFO:teuthology.orchestra.run.smithi090.stdout:(31/166): ceph-mgr-dashboard-19.0.0-1514.g25296 2.1 MB/s | 1.8 MB 00:00 2024-02-26T06:09:46.764 INFO:teuthology.orchestra.run.smithi090.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 1.6 MB/s | 247 kB 00:00 2024-02-26T06:09:46.848 INFO:teuthology.orchestra.run.smithi090.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 626 kB/s | 52 kB 00:00 2024-02-26T06:09:46.931 INFO:teuthology.orchestra.run.smithi090.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1514.g2 184 kB/s | 15 kB 00:00 2024-02-26T06:09:47.099 INFO:teuthology.orchestra.run.smithi090.stdout:(35/166): ceph-volume-19.0.0-1514.g25296a8f.el9 1.5 MB/s | 264 kB 00:00 2024-02-26T06:09:47.120 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-02-26T06:09:47.365 INFO:teuthology.orchestra.run.smithi090.stdout:(36/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 2.7 MB/s | 748 kB 00:00 2024-02-26T06:09:47.458 INFO:teuthology.orchestra.run.smithi090.stdout:(37/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 199 kB/s | 18 kB 00:00 2024-02-26T06:09:47.574 INFO:teuthology.orchestra.run.smithi090.stdout:(38/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 151 kB/s | 18 kB 00:00 2024-02-26T06:09:47.683 INFO:teuthology.orchestra.run.smithi090.stdout:(39/166): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-02-26T06:09:47.750 INFO:teuthology.orchestra.run.smithi090.stdout:(40/166): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-02-26T06:09:47.834 INFO:teuthology.orchestra.run.smithi090.stdout:(41/166): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-02-26T06:09:47.884 INFO:teuthology.orchestra.run.smithi090.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-02-26T06:09:47.934 INFO:teuthology.orchestra.run.smithi090.stdout:(43/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 218 kB 00:00 2024-02-26T06:09:47.985 INFO:teuthology.orchestra.run.smithi090.stdout:(44/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-02-26T06:09:48.069 INFO:teuthology.orchestra.run.smithi090.stdout:(45/166): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-02-26T06:09:48.119 INFO:teuthology.orchestra.run.smithi090.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-02-26T06:09:48.169 INFO:teuthology.orchestra.run.smithi090.stdout:(47/166): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-02-26T06:09:48.219 INFO:teuthology.orchestra.run.smithi090.stdout:(48/166): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-02-26T06:09:48.270 INFO:teuthology.orchestra.run.smithi090.stdout:(49/166): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-02-26T06:09:48.563 INFO:teuthology.orchestra.run.smithi090.stdout:(50/166): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-02-26T06:09:48.680 INFO:teuthology.orchestra.run.smithi090.stdout:(51/166): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-02-26T06:09:48.739 INFO:teuthology.orchestra.run.smithi090.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-02-26T06:09:48.806 INFO:teuthology.orchestra.run.smithi090.stdout:(53/166): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-02-26T06:09:48.865 INFO:teuthology.orchestra.run.smithi090.stdout:(54/166): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-02-26T06:09:48.923 INFO:teuthology.orchestra.run.smithi090.stdout:(55/166): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-02-26T06:09:49.082 INFO:teuthology.orchestra.run.smithi090.stdout:(56/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-02-26T06:09:49.132 INFO:teuthology.orchestra.run.smithi090.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-02-26T06:09:49.191 INFO:teuthology.orchestra.run.smithi090.stdout:(58/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-02-26T06:09:49.251 INFO:teuthology.orchestra.run.smithi090.stdout:(59/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-02-26T06:09:49.326 INFO:teuthology.orchestra.run.smithi090.stdout:(60/166): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-02-26T06:09:49.393 INFO:teuthology.orchestra.run.smithi090.stdout:(61/166): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-02-26T06:09:49.452 INFO:teuthology.orchestra.run.smithi090.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 435 kB/s | 26 kB 00:00 2024-02-26T06:09:49.511 INFO:teuthology.orchestra.run.smithi090.stdout:(63/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-02-26T06:09:49.578 INFO:teuthology.orchestra.run.smithi090.stdout:(64/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-02-26T06:09:49.645 INFO:teuthology.orchestra.run.smithi090.stdout:(65/166): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-02-26T06:09:49.787 INFO:teuthology.orchestra.run.smithi090.stdout:(66/166): openblas-0.3.21-2.el9.x86_64.rpm 248 kB/s | 35 kB 00:00 2024-02-26T06:09:49.921 INFO:teuthology.orchestra.run.smithi006.stderr:Modular dependency problems: 2024-02-26T06:09:49.922 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-02-26T06:09:49.922 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 1: conflicting requests 2024-02-26T06:09:49.922 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.922 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 2: conflicting requests 2024-02-26T06:09:49.922 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.922 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 3: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 4: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 5: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 6: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 7: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 8: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 9: conflicting requests 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.923 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 10: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 11: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 12: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 13: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 14: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 15: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 16: conflicting requests 2024-02-26T06:09:49.924 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 17: conflicting requests 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 18: conflicting requests 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 19: conflicting requests 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 20: conflicting requests 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 21: conflicting requests 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 22: conflicting requests 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.925 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 23: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 24: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 25: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 26: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 27: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 28: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 29: conflicting requests 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.926 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 30: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 31: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 32: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 33: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 34: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 35: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 36: conflicting requests 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.927 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 38: conflicting requests 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.928 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.929 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.930 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.931 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T06:09:49.931 INFO:teuthology.orchestra.run.smithi006.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T06:09:50.004 INFO:teuthology.orchestra.run.smithi090.stdout:(67/166): ceph-mgr-diskprediction-local-19.0.0- 1.8 MB/s | 7.4 MB 00:04 2024-02-26T06:09:50.080 INFO:teuthology.orchestra.run.smithi090.stdout:(68/166): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-02-26T06:09:50.497 INFO:teuthology.orchestra.run.smithi090.stdout:(69/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 8.4 MB/s | 4.1 MB 00:00 2024-02-26T06:09:50.519 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:09:50.521 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:09:50.543 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T06:09:50.544 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T06:09:50.546 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T06:09:50.547 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T06:09:50.550 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T06:09:50.554 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T06:09:50.555 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T06:09:50.588 INFO:teuthology.orchestra.run.smithi090.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 9.7 MB/s | 905 kB 00:00 2024-02-26T06:09:50.647 INFO:teuthology.orchestra.run.smithi090.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 135 kB 00:00 2024-02-26T06:09:50.660 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-02-26T06:09:50.679 INFO:teuthology.orchestra.run.smithi006.stdout:========================================================================================= 2024-02-26T06:09:50.679 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-02-26T06:09:50.679 INFO:teuthology.orchestra.run.smithi006.stdout:========================================================================================= 2024-02-26T06:09:50.679 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T06:09:50.680 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T06:09:50.681 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T06:09:50.682 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T06:09:50.683 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-02-26T06:09:50.684 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T06:09:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T06:09:50.686 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T06:09:50.687 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T06:09:50.688 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T06:09:50.689 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T06:09:50.690 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T06:09:50.691 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T06:09:50.692 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T06:09:50.693 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout:========================================================================================= 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout:Install 150 Packages 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 16 Packages 2024-02-26T06:09:50.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 221 M 2024-02-26T06:09:50.695 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-02-26T06:09:50.915 INFO:teuthology.orchestra.run.smithi090.stdout:(72/166): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-02-26T06:09:50.998 INFO:teuthology.orchestra.run.smithi090.stdout:(73/166): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-02-26T06:09:51.057 INFO:teuthology.orchestra.run.smithi090.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-02-26T06:09:51.290 INFO:teuthology.orchestra.run.smithi090.stdout:(75/166): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-02-26T06:09:51.349 INFO:teuthology.orchestra.run.smithi090.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-02-26T06:09:51.409 INFO:teuthology.orchestra.run.smithi090.stdout:(77/166): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-02-26T06:09:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-02-26T06:09:51.526 INFO:teuthology.orchestra.run.smithi090.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-02-26T06:09:51.585 INFO:teuthology.orchestra.run.smithi090.stdout:(80/166): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-02-26T06:09:51.644 INFO:teuthology.orchestra.run.smithi090.stdout:(81/166): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-02-26T06:09:51.747 INFO:teuthology.orchestra.run.smithi006.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-02-26T06:09:51.836 INFO:teuthology.orchestra.run.smithi090.stdout:(82/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-02-26T06:09:52.122 INFO:teuthology.orchestra.run.smithi006.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 2.3 MB/s | 886 kB 00:00 2024-02-26T06:09:52.205 INFO:teuthology.orchestra.run.smithi006.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 1.7 MB/s | 145 kB 00:00 2024-02-26T06:09:52.561 INFO:teuthology.orchestra.run.smithi090.stdout:(83/166): postgresql-13.11-1.el9.x86_64.rpm 653 kB/s | 1.6 MB 00:02 2024-02-26T06:09:52.620 INFO:teuthology.orchestra.run.smithi090.stdout:(84/166): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-02-26T06:09:52.687 INFO:teuthology.orchestra.run.smithi090.stdout:(85/166): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-02-26T06:09:52.739 INFO:teuthology.orchestra.run.smithi006.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 4.0 MB/s | 2.1 MB 00:00 2024-02-26T06:09:52.745 INFO:teuthology.orchestra.run.smithi090.stdout:(86/166): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-02-26T06:09:52.813 INFO:teuthology.orchestra.run.smithi090.stdout:(87/166): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-02-26T06:09:52.888 INFO:teuthology.orchestra.run.smithi090.stdout:(88/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-02-26T06:09:52.955 INFO:teuthology.orchestra.run.smithi090.stdout:(89/166): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-02-26T06:09:52.964 INFO:teuthology.orchestra.run.smithi006.stdout:(5/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-02-26T06:09:53.014 INFO:teuthology.orchestra.run.smithi090.stdout:(90/166): python3-pytz-2021.1-5.el9.noarch.rpm 876 kB/s | 51 kB 00:00 2024-02-26T06:09:53.072 INFO:teuthology.orchestra.run.smithi090.stdout:(91/166): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-02-26T06:09:53.540 INFO:teuthology.orchestra.run.smithi006.stdout:(6/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-02-26T06:09:53.692 INFO:teuthology.orchestra.run.smithi006.stdout:(7/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 2.4 MB/s | 5.3 MB 00:02 2024-02-26T06:09:53.873 INFO:teuthology.orchestra.run.smithi090.stdout:(92/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 123 kB/s | 250 kB 00:02 2024-02-26T06:09:53.923 INFO:teuthology.orchestra.run.smithi090.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-02-26T06:09:53.973 INFO:teuthology.orchestra.run.smithi090.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-02-26T06:09:54.040 INFO:teuthology.orchestra.run.smithi090.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-02-26T06:09:54.091 INFO:teuthology.orchestra.run.smithi090.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 698 kB/s | 35 kB 00:00 2024-02-26T06:09:54.149 INFO:teuthology.orchestra.run.smithi090.stdout:(97/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 761 kB/s | 44 kB 00:00 2024-02-26T06:09:54.217 INFO:teuthology.orchestra.run.smithi090.stdout:(98/166): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-02-26T06:09:54.275 INFO:teuthology.orchestra.run.smithi090.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-02-26T06:09:54.344 INFO:teuthology.orchestra.run.smithi090.stdout:(100/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.3 MB/s | 94 kB 00:00 2024-02-26T06:09:54.402 INFO:teuthology.orchestra.run.smithi090.stdout:(101/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-02-26T06:09:55.134 INFO:teuthology.orchestra.run.smithi090.stdout:(102/166): ceph-test-19.0.0-1514.g25296a8f.el9. 3.1 MB/s | 47 MB 00:15 2024-02-26T06:09:55.162 INFO:teuthology.orchestra.run.smithi090.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 59 kB/s | 45 kB 00:00 2024-02-26T06:09:55.273 INFO:teuthology.orchestra.run.smithi006.stdout:(8/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x86 9.7 MB/s | 17 MB 00:01 2024-02-26T06:09:55.332 INFO:teuthology.orchestra.run.smithi006.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 436 kB/s | 25 kB 00:00 2024-02-26T06:09:55.362 INFO:teuthology.orchestra.run.smithi090.stdout:(104/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.1 MB/s | 16 MB 00:02 2024-02-26T06:09:55.499 INFO:teuthology.orchestra.run.smithi006.stdout:(10/166): ceph-radosgw-19.0.0-1514.g25296a8f.el 5.4 MB/s | 9.7 MB 00:01 2024-02-26T06:09:55.571 INFO:teuthology.orchestra.run.smithi090.stdout:(105/166): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2024-02-26T06:09:55.582 INFO:teuthology.orchestra.run.smithi006.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 2.6 MB/s | 217 kB 00:00 2024-02-26T06:09:55.596 INFO:teuthology.orchestra.run.smithi090.stdout:(106/166): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-02-26T06:09:55.630 INFO:teuthology.orchestra.run.smithi090.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 582 kB/s | 19 kB 00:00 2024-02-26T06:09:55.714 INFO:teuthology.orchestra.run.smithi090.stdout:(108/166): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-02-26T06:09:55.756 INFO:teuthology.orchestra.run.smithi090.stdout:(109/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 623 kB/s | 26 kB 00:00 2024-02-26T06:09:55.840 INFO:teuthology.orchestra.run.smithi090.stdout:(110/166): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-02-26T06:09:55.858 INFO:teuthology.orchestra.run.smithi006.stdout:(12/166): ceph-common-19.0.0-1514.g25296a8f.el9 4.5 MB/s | 20 MB 00:04 2024-02-26T06:09:55.865 INFO:teuthology.orchestra.run.smithi090.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-02-26T06:09:55.883 INFO:teuthology.orchestra.run.smithi006.stdout:(13/166): cephfs-shell-19.0.0-1514.g25296a8f.el 90 kB/s | 27 kB 00:00 2024-02-26T06:09:55.890 INFO:teuthology.orchestra.run.smithi090.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-02-26T06:09:55.916 INFO:teuthology.orchestra.run.smithi090.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-02-26T06:09:55.933 INFO:teuthology.orchestra.run.smithi006.stdout:(14/166): libcephfs-devel-19.0.0-1514.g25296a8f 421 kB/s | 31 kB 00:00 2024-02-26T06:09:55.958 INFO:teuthology.orchestra.run.smithi090.stdout:(114/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-02-26T06:09:55.992 INFO:teuthology.orchestra.run.smithi090.stdout:(115/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-02-26T06:09:56.033 INFO:teuthology.orchestra.run.smithi090.stdout:(116/166): protobuf-compiler-3.14.0-13.el9.x86_ 991 kB/s | 863 kB 00:00 2024-02-26T06:09:56.034 INFO:teuthology.orchestra.run.smithi006.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.6 MB/s | 163 kB 00:00 2024-02-26T06:09:56.059 INFO:teuthology.orchestra.run.smithi090.stdout:(117/166): python3-asyncssh-2.13.2-4.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-02-26T06:09:56.084 INFO:teuthology.orchestra.run.smithi006.stdout:(16/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 3.6 MB/s | 743 kB 00:00 2024-02-26T06:09:56.085 INFO:teuthology.orchestra.run.smithi090.stdout:(118/166): python3-cachetools-4.2.4-1.el9.noarc 344 kB/s | 32 kB 00:00 2024-02-26T06:09:56.111 INFO:teuthology.orchestra.run.smithi090.stdout:(119/166): python3-certifi-2021.10.8-2.el9.noar 195 kB/s | 15 kB 00:00 2024-02-26T06:09:56.117 INFO:teuthology.orchestra.run.smithi006.stdout:(17/166): librados-devel-19.0.0-1514.g25296a8f. 1.5 MB/s | 127 kB 00:00 2024-02-26T06:09:56.144 INFO:teuthology.orchestra.run.smithi090.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-02-26T06:09:56.178 INFO:teuthology.orchestra.run.smithi090.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-02-26T06:09:56.203 INFO:teuthology.orchestra.run.smithi090.stdout:(122/166): python3-colorama-0.4.4-12.el9.noarch 602 kB/s | 35 kB 00:00 2024-02-26T06:09:56.234 INFO:teuthology.orchestra.run.smithi006.stdout:(18/166): libradosstriper1-19.0.0-1514.g25296a8 3.3 MB/s | 498 kB 00:00 2024-02-26T06:09:56.237 INFO:teuthology.orchestra.run.smithi090.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-02-26T06:09:56.270 INFO:teuthology.orchestra.run.smithi090.stdout:(124/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-02-26T06:09:56.296 INFO:teuthology.orchestra.run.smithi090.stdout:(125/166): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-02-26T06:09:56.310 INFO:teuthology.orchestra.run.smithi006.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 609 kB/s | 45 kB 00:00 2024-02-26T06:09:56.321 INFO:teuthology.orchestra.run.smithi090.stdout:(126/166): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-02-26T06:09:56.347 INFO:teuthology.orchestra.run.smithi090.stdout:(127/166): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-02-26T06:09:56.372 INFO:teuthology.orchestra.run.smithi090.stdout:(128/166): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-02-26T06:09:56.406 INFO:teuthology.orchestra.run.smithi090.stdout:(129/166): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-02-26T06:09:56.410 INFO:teuthology.orchestra.run.smithi006.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 1.3 MB/s | 131 kB 00:00 2024-02-26T06:09:56.431 INFO:teuthology.orchestra.run.smithi090.stdout:(130/166): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-02-26T06:09:56.490 INFO:teuthology.orchestra.run.smithi090.stdout:(131/166): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-02-26T06:09:56.502 INFO:teuthology.orchestra.run.smithi006.stdout:(21/166): python3-cephfs-19.0.0-1514.g25296a8f. 1.7 MB/s | 161 kB 00:00 2024-02-26T06:09:56.515 INFO:teuthology.orchestra.run.smithi090.stdout:(132/166): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-02-26T06:09:56.549 INFO:teuthology.orchestra.run.smithi090.stdout:(133/166): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-02-26T06:09:56.574 INFO:teuthology.orchestra.run.smithi090.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 939 kB/s | 79 kB 00:00 2024-02-26T06:09:56.599 INFO:teuthology.orchestra.run.smithi090.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 686 kB/s | 58 kB 00:00 2024-02-26T06:09:56.611 INFO:teuthology.orchestra.run.smithi006.stdout:(22/166): python3-rados-19.0.0-1514.g25296a8f.e 2.9 MB/s | 320 kB 00:00 2024-02-26T06:09:56.625 INFO:teuthology.orchestra.run.smithi090.stdout:(136/166): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-02-26T06:09:56.658 INFO:teuthology.orchestra.run.smithi090.stdout:(137/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-02-26T06:09:56.684 INFO:teuthology.orchestra.run.smithi090.stdout:(138/166): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-02-26T06:09:56.709 INFO:teuthology.orchestra.run.smithi090.stdout:(139/166): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-02-26T06:09:56.720 INFO:teuthology.orchestra.run.smithi006.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 2.7 MB/s | 298 kB 00:00 2024-02-26T06:09:56.735 INFO:teuthology.orchestra.run.smithi090.stdout:(140/166): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-02-26T06:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout:(141/166): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-02-26T06:09:56.787 INFO:teuthology.orchestra.run.smithi090.stdout:(142/166): python3-tempora-5.0.0-2.el9.noarch.r 692 kB/s | 36 kB 00:00 2024-02-26T06:09:56.812 INFO:teuthology.orchestra.run.smithi090.stdout:(143/166): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-02-26T06:09:56.820 INFO:teuthology.orchestra.run.smithi006.stdout:(24/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 7.3 MB/s | 5.1 MB 00:00 2024-02-26T06:09:56.837 INFO:teuthology.orchestra.run.smithi090.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 692 kB/s | 51 kB 00:00 2024-02-26T06:09:56.853 INFO:teuthology.orchestra.run.smithi006.stdout:(25/166): python3-rgw-19.0.0-1514.g25296a8f.el9 749 kB/s | 99 kB 00:00 2024-02-26T06:09:56.862 INFO:teuthology.orchestra.run.smithi090.stdout:(145/166): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-02-26T06:09:56.888 INFO:teuthology.orchestra.run.smithi090.stdout:(146/166): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-26T06:09:56.904 INFO:teuthology.orchestra.run.smithi006.stdout:(26/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-26T06:09:56.913 INFO:teuthology.orchestra.run.smithi090.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-02-26T06:09:56.939 INFO:teuthology.orchestra.run.smithi090.stdout:(148/166): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-02-26T06:09:56.964 INFO:teuthology.orchestra.run.smithi090.stdout:(149/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-02-26T06:09:56.996 INFO:teuthology.orchestra.run.smithi006.stdout:(27/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 1.8 MB/s | 171 kB 00:00 2024-02-26T06:09:57.014 INFO:teuthology.orchestra.run.smithi090.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-02-26T06:09:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 347 kB/s | 31 kB 00:00 2024-02-26T06:09:57.188 INFO:teuthology.orchestra.run.smithi006.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 1.4 MB/s | 138 kB 00:00 2024-02-26T06:09:57.290 INFO:teuthology.orchestra.run.smithi090.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 8.3 MB/s | 2.3 MB 00:00 2024-02-26T06:09:57.388 INFO:teuthology.orchestra.run.smithi006.stdout:(30/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 5.8 MB/s | 3.1 MB 00:00 2024-02-26T06:09:57.432 INFO:teuthology.orchestra.run.smithi090.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 13 MB/s | 1.8 MB 00:00 2024-02-26T06:09:57.490 INFO:teuthology.orchestra.run.smithi090.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 725 kB/s | 42 kB 00:00 2024-02-26T06:09:57.522 INFO:teuthology.orchestra.run.smithi006.stdout:(31/166): ceph-mgr-dashboard-19.0.0-1514.g25296 5.4 MB/s | 1.8 MB 00:00 2024-02-26T06:09:57.566 INFO:teuthology.orchestra.run.smithi090.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 3.2 MB/s | 247 kB 00:00 2024-02-26T06:09:57.623 INFO:teuthology.orchestra.run.smithi006.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 2.4 MB/s | 247 kB 00:00 2024-02-26T06:09:57.624 INFO:teuthology.orchestra.run.smithi090.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-02-26T06:09:57.683 INFO:teuthology.orchestra.run.smithi090.stdout:(156/166): elfutils-debuginfod-client-0.190-2.e 623 kB/s | 36 kB 00:00 2024-02-26T06:09:57.748 INFO:teuthology.orchestra.run.smithi006.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 421 kB/s | 52 kB 00:00 2024-02-26T06:09:57.758 INFO:teuthology.orchestra.run.smithi090.stdout:(157/166): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-02-26T06:09:57.844 INFO:teuthology.orchestra.run.smithi090.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-02-26T06:09:57.856 INFO:teuthology.orchestra.run.smithi006.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1514.g2 143 kB/s | 15 kB 00:00 2024-02-26T06:09:57.901 INFO:teuthology.orchestra.run.smithi090.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 396 kB/s | 23 kB 00:00 2024-02-26T06:09:57.976 INFO:teuthology.orchestra.run.smithi090.stdout:(160/166): openssl-3.0.7-25.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-02-26T06:09:57.982 INFO:teuthology.orchestra.run.smithi006.stdout:(35/166): ceph-volume-19.0.0-1514.g25296a8f.el9 2.1 MB/s | 264 kB 00:00 2024-02-26T06:09:58.085 INFO:teuthology.orchestra.run.smithi090.stdout:(161/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 20 MB/s | 2.2 MB 00:00 2024-02-26T06:09:58.143 INFO:teuthology.orchestra.run.smithi090.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-02-26T06:09:58.191 INFO:teuthology.orchestra.run.smithi006.stdout:(36/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 3.5 MB/s | 748 kB 00:00 2024-02-26T06:09:58.210 INFO:teuthology.orchestra.run.smithi090.stdout:(163/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-02-26T06:09:58.358 INFO:teuthology.orchestra.run.smithi006.stdout:(37/166): ceph-mgr-diskprediction-local-19.0.0- 7.7 MB/s | 7.4 MB 00:00 2024-02-26T06:09:58.382 INFO:teuthology.orchestra.run.smithi006.stdout:(38/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 96 kB/s | 18 kB 00:00 2024-02-26T06:09:58.436 INFO:teuthology.orchestra.run.smithi090.stdout:(164/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-02-26T06:09:58.503 INFO:teuthology.orchestra.run.smithi090.stdout:(165/166): librados2-19.0.0-1514.g25296a8f.el9. 2.2 MB/s | 3.4 MB 00:01 2024-02-26T06:09:58.525 INFO:teuthology.orchestra.run.smithi006.stdout:(39/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 106 kB/s | 18 kB 00:00 2024-02-26T06:09:58.528 INFO:teuthology.orchestra.run.smithi090.stdout:(166/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 234 kB/s | 74 kB 00:00 2024-02-26T06:09:58.531 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:09:58.531 INFO:teuthology.orchestra.run.smithi090.stdout:Total 8.6 MB/s | 221 MB 00:25 2024-02-26T06:09:59.117 INFO:teuthology.orchestra.run.smithi006.stdout:(40/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 12 MB/s | 47 MB 00:03 2024-02-26T06:09:59.142 INFO:teuthology.orchestra.run.smithi006.stdout:(41/166): fuse-2.9.9-15.el9.x86_64.rpm 106 kB/s | 80 kB 00:00 2024-02-26T06:09:59.192 INFO:teuthology.orchestra.run.smithi006.stdout:(42/166): jq-1.6-16.el9.x86_64.rpm 281 kB/s | 187 kB 00:00 2024-02-26T06:09:59.259 INFO:teuthology.orchestra.run.smithi006.stdout:(43/166): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-02-26T06:09:59.293 INFO:teuthology.orchestra.run.smithi006.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 218 kB 00:00 2024-02-26T06:09:59.360 INFO:teuthology.orchestra.run.smithi006.stdout:(45/166): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-02-26T06:09:59.402 INFO:teuthology.orchestra.run.smithi006.stdout:(46/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-02-26T06:09:59.427 INFO:teuthology.orchestra.run.smithi006.stdout:(47/166): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-02-26T06:09:59.461 INFO:teuthology.orchestra.run.smithi006.stdout:(48/166): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-02-26T06:09:59.495 INFO:teuthology.orchestra.run.smithi006.stdout:(49/166): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-02-26T06:09:59.537 INFO:teuthology.orchestra.run.smithi006.stdout:(50/166): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-02-26T06:09:59.794 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-02-26T06:09:59.870 INFO:teuthology.orchestra.run.smithi006.stdout:(51/166): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-02-26T06:09:59.921 INFO:teuthology.orchestra.run.smithi006.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-02-26T06:09:59.971 INFO:teuthology.orchestra.run.smithi006.stdout:(53/166): elfutils-libelf-devel-0.190-2.el9.x86 437 kB/s | 21 kB 00:00 2024-02-26T06:10:00.051 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-02-26T06:10:00.051 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-02-26T06:10:00.071 INFO:teuthology.orchestra.run.smithi006.stdout:(54/166): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 943 kB 00:00 2024-02-26T06:10:00.105 INFO:teuthology.orchestra.run.smithi006.stdout:(55/166): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-02-26T06:10:00.130 INFO:teuthology.orchestra.run.smithi006.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 566 kB/s | 33 kB 00:00 2024-02-26T06:10:00.189 INFO:teuthology.orchestra.run.smithi006.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 317 kB/s | 18 kB 00:00 2024-02-26T06:10:00.256 INFO:teuthology.orchestra.run.smithi006.stdout:(58/166): python3-urllib3-1.26.5-5.el9.noarch.r 270 kB/s | 215 kB 00:00 2024-02-26T06:10:00.289 INFO:teuthology.orchestra.run.smithi006.stdout:(59/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-02-26T06:10:00.381 INFO:teuthology.orchestra.run.smithi006.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 364 kB/s | 45 kB 00:00 2024-02-26T06:10:00.506 INFO:teuthology.orchestra.run.smithi006.stdout:(61/166): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-02-26T06:10:00.565 INFO:teuthology.orchestra.run.smithi006.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 442 kB/s | 26 kB 00:00 2024-02-26T06:10:00.649 INFO:teuthology.orchestra.run.smithi006.stdout:(63/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.5 MB/s | 3.0 MB 00:00 2024-02-26T06:10:00.674 INFO:teuthology.orchestra.run.smithi006.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 441 kB/s | 47 kB 00:00 2024-02-26T06:10:00.732 INFO:teuthology.orchestra.run.smithi006.stdout:(65/166): libtool-2.4.6-45.el9.x86_64.rpm 1.6 MB/s | 582 kB 00:00 2024-02-26T06:10:00.766 INFO:teuthology.orchestra.run.smithi006.stdout:(66/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-02-26T06:10:00.791 INFO:teuthology.orchestra.run.smithi006.stdout:(67/166): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-02-26T06:10:00.816 INFO:teuthology.orchestra.run.smithi006.stdout:(68/166): openblas-0.3.21-2.el9.x86_64.rpm 425 kB/s | 35 kB 00:00 2024-02-26T06:10:01.075 INFO:teuthology.orchestra.run.smithi006.stdout:(69/166): postgresql-13.11-1.el9.x86_64.rpm 6.2 MB/s | 1.6 MB 00:00 2024-02-26T06:10:01.192 INFO:teuthology.orchestra.run.smithi006.stdout:(70/166): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-02-26T06:10:01.242 INFO:teuthology.orchestra.run.smithi006.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 135 kB 00:00 2024-02-26T06:10:01.343 INFO:teuthology.orchestra.run.smithi006.stdout:(72/166): postgresql-contrib-13.11-1.el9.x86_64 3.3 MB/s | 905 kB 00:00 2024-02-26T06:10:01.443 INFO:teuthology.orchestra.run.smithi006.stdout:(73/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 6.6 MB/s | 4.1 MB 00:00 2024-02-26T06:10:01.493 INFO:teuthology.orchestra.run.smithi006.stdout:(74/166): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-02-26T06:10:01.519 INFO:teuthology.orchestra.run.smithi006.stdout:(75/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-02-26T06:10:01.577 INFO:teuthology.orchestra.run.smithi006.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-02-26T06:10:01.669 INFO:teuthology.orchestra.run.smithi006.stdout:(77/166): postgresql-server-13.11-1.el9.x86_64. 13 MB/s | 5.7 MB 00:00 2024-02-26T06:10:01.695 INFO:teuthology.orchestra.run.smithi006.stdout:(78/166): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-02-26T06:10:01.720 INFO:teuthology.orchestra.run.smithi006.stdout:(79/166): python3-jsonpatch-1.21-16.el9.noarch. 518 kB/s | 26 kB 00:00 2024-02-26T06:10:01.745 INFO:teuthology.orchestra.run.smithi006.stdout:(80/166): python3-jsonpointer-2.0-4.el9.noarch. 386 kB/s | 19 kB 00:00 2024-02-26T06:10:01.779 INFO:teuthology.orchestra.run.smithi006.stdout:(81/166): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-02-26T06:10:01.822 INFO:teuthology.orchestra.run.smithi006.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-02-26T06:10:01.827 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-02-26T06:10:01.827 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-02-26T06:10:01.905 INFO:teuthology.orchestra.run.smithi006.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.0 MB/s | 250 kB 00:00 2024-02-26T06:10:01.972 INFO:teuthology.orchestra.run.smithi006.stdout:(84/166): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-02-26T06:10:02.030 INFO:teuthology.orchestra.run.smithi006.stdout:(85/166): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-02-26T06:10:02.147 INFO:teuthology.orchestra.run.smithi006.stdout:(86/166): python3-babel-2.9.1-2.el9.noarch.rpm 9.1 MB/s | 6.0 MB 00:00 2024-02-26T06:10:02.172 INFO:teuthology.orchestra.run.smithi006.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 299 kB/s | 42 kB 00:00 2024-02-26T06:10:02.206 INFO:teuthology.orchestra.run.smithi006.stdout:(88/166): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-02-26T06:10:02.240 INFO:teuthology.orchestra.run.smithi006.stdout:(89/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-02-26T06:10:02.340 INFO:teuthology.orchestra.run.smithi006.stdout:(90/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.1 MB/s | 5.1 MB 00:00 2024-02-26T06:10:02.366 INFO:teuthology.orchestra.run.smithi006.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 410 kB/s | 51 kB 00:00 2024-02-26T06:10:02.391 INFO:teuthology.orchestra.run.smithi006.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-02-26T06:10:02.416 INFO:teuthology.orchestra.run.smithi006.stdout:(93/166): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-02-26T06:10:02.466 INFO:teuthology.orchestra.run.smithi006.stdout:(94/166): python3-toml-0.10.2-6.el9.noarch.rpm 558 kB/s | 42 kB 00:00 2024-02-26T06:10:02.491 INFO:teuthology.orchestra.run.smithi006.stdout:(95/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 592 kB/s | 44 kB 00:00 2024-02-26T06:10:02.533 INFO:teuthology.orchestra.run.smithi006.stdout:(96/166): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-02-26T06:10:02.552 INFO:teuthology.orchestra.run.smithi006.stdout:(97/166): qatlib-service-23.11.0-1.el9.x86_64.r 602 kB/s | 35 kB 00:00 2024-02-26T06:10:02.575 INFO:teuthology.orchestra.run.smithi006.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-02-26T06:10:02.609 INFO:teuthology.orchestra.run.smithi006.stdout:(99/166): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-02-26T06:10:02.635 INFO:teuthology.orchestra.run.smithi006.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-02-26T06:10:02.660 INFO:teuthology.orchestra.run.smithi006.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-02-26T06:10:02.685 INFO:teuthology.orchestra.run.smithi006.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-02-26T06:10:02.719 INFO:teuthology.orchestra.run.smithi006.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 790 kB/s | 45 kB 00:00 2024-02-26T06:10:02.919 INFO:teuthology.orchestra.run.smithi006.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 96 kB/s | 22 kB 00:00 2024-02-26T06:10:03.161 INFO:teuthology.orchestra.run.smithi006.stdout:(105/166): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-02-26T06:10:03.203 INFO:teuthology.orchestra.run.smithi006.stdout:(106/166): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-02-26T06:10:03.237 INFO:teuthology.orchestra.run.smithi006.stdout:(107/166): gperftools-libs-2.9.1-2.el9.x86_64.r 4.1 MB/s | 309 kB 00:00 2024-02-26T06:10:03.262 INFO:teuthology.orchestra.run.smithi006.stdout:(108/166): grpc-data-1.46.7-10.el9.noarch.rpm 353 kB/s | 19 kB 00:00 2024-02-26T06:10:03.454 INFO:teuthology.orchestra.run.smithi006.stdout:(109/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-02-26T06:10:03.496 INFO:teuthology.orchestra.run.smithi006.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 111 kB/s | 26 kB 00:00 2024-02-26T06:10:03.625 INFO:teuthology.orchestra.run.smithi006.stdout:(111/166): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-02-26T06:10:03.655 INFO:teuthology.orchestra.run.smithi006.stdout:(112/166): liboath-2.6.7-2.el9.x86_64.rpm 261 kB/s | 49 kB 00:00 2024-02-26T06:10:03.680 INFO:teuthology.orchestra.run.smithi006.stdout:(113/166): libunwind-1.6.2-1.el9.x86_64.rpm 398 kB/s | 67 kB 00:00 2024-02-26T06:10:03.722 INFO:teuthology.orchestra.run.smithi006.stdout:(114/166): luarocks-3.9.2-1.el9.noarch.rpm 2.3 MB/s | 151 kB 00:00 2024-02-26T06:10:03.756 INFO:teuthology.orchestra.run.smithi006.stdout:(115/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.3 MB/s | 838 kB 00:00 2024-02-26T06:10:03.806 INFO:teuthology.orchestra.run.smithi006.stdout:(116/166): python3-asyncssh-2.13.2-4.el9.noarch 4.7 MB/s | 548 kB 00:00 2024-02-26T06:10:03.840 INFO:teuthology.orchestra.run.smithi006.stdout:(117/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 562 kB/s | 43 kB 00:00 2024-02-26T06:10:03.865 INFO:teuthology.orchestra.run.smithi006.stdout:(118/166): python3-cachetools-4.2.4-1.el9.noarc 315 kB/s | 32 kB 00:00 2024-02-26T06:10:03.890 INFO:teuthology.orchestra.run.smithi006.stdout:(119/166): python3-certifi-2021.10.8-2.el9.noar 183 kB/s | 15 kB 00:00 2024-02-26T06:10:03.924 INFO:teuthology.orchestra.run.smithi006.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-02-26T06:10:03.949 INFO:teuthology.orchestra.run.smithi006.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 4.3 MB/s | 358 kB 00:00 2024-02-26T06:10:03.982 INFO:teuthology.orchestra.run.smithi006.stdout:(122/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.3 MB/s | 303 kB 00:00 2024-02-26T06:10:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 3.2 MB/s | 203 kB 00:00 2024-02-26T06:10:04.050 INFO:teuthology.orchestra.run.smithi006.stdout:(124/166): python3-colorama-0.4.4-12.el9.noarch 365 kB/s | 35 kB 00:00 2024-02-26T06:10:04.125 INFO:teuthology.orchestra.run.smithi006.stdout:(125/166): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-02-26T06:10:04.157 INFO:teuthology.orchestra.run.smithi006.stdout:(126/166): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-02-26T06:10:04.207 INFO:teuthology.orchestra.run.smithi006.stdout:(127/166): python3-jaraco-8.2.1-3.el9.noarch.rp 73 kB/s | 11 kB 00:00 2024-02-26T06:10:04.233 INFO:teuthology.orchestra.run.smithi006.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 239 kB/s | 18 kB 00:00 2024-02-26T06:10:04.258 INFO:teuthology.orchestra.run.smithi006.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 234 kB/s | 23 kB 00:00 2024-02-26T06:10:04.275 INFO:teuthology.orchestra.run.smithi006.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 294 kB/s | 19 kB 00:00 2024-02-26T06:10:04.300 INFO:teuthology.orchestra.run.smithi006.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 482 kB/s | 20 kB 00:00 2024-02-26T06:10:04.351 INFO:teuthology.orchestra.run.smithi006.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-02-26T06:10:04.367 INFO:teuthology.orchestra.run.smithi006.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-02-26T06:10:04.384 INFO:teuthology.orchestra.run.smithi006.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-02-26T06:10:04.418 INFO:teuthology.orchestra.run.smithi006.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 1.2 MB/s | 58 kB 00:00 2024-02-26T06:10:04.451 INFO:teuthology.orchestra.run.smithi006.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 3.3 MB/s | 272 kB 00:00 2024-02-26T06:10:04.477 INFO:teuthology.orchestra.run.smithi006.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-02-26T06:10:04.514 INFO:teuthology.orchestra.run.smithi006.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-02-26T06:10:04.527 INFO:teuthology.orchestra.run.smithi006.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 382 kB/s | 28 kB 00:00 2024-02-26T06:10:04.553 INFO:teuthology.orchestra.run.smithi006.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 419 kB/s | 31 kB 00:00 2024-02-26T06:10:04.578 INFO:teuthology.orchestra.run.smithi006.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 3.8 MB/s | 188 kB 00:00 2024-02-26T06:10:04.603 INFO:teuthology.orchestra.run.smithi006.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 801 kB/s | 59 kB 00:00 2024-02-26T06:10:04.628 INFO:teuthology.orchestra.run.smithi006.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 495 kB/s | 36 kB 00:00 2024-02-26T06:10:04.653 INFO:teuthology.orchestra.run.smithi006.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-02-26T06:10:04.679 INFO:teuthology.orchestra.run.smithi006.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-26T06:10:04.704 INFO:teuthology.orchestra.run.smithi006.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-02-26T06:10:04.729 INFO:teuthology.orchestra.run.smithi006.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 416 kB/s | 20 kB 00:00 2024-02-26T06:10:04.779 INFO:teuthology.orchestra.run.smithi006.stdout:(148/166): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-02-26T06:10:04.805 INFO:teuthology.orchestra.run.smithi006.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-26T06:10:04.872 INFO:teuthology.orchestra.run.smithi006.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-02-26T06:10:06.088 INFO:teuthology.orchestra.run.smithi006.stdout:(151/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-02-26T06:10:06.306 INFO:teuthology.orchestra.run.smithi006.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 8.4 MB/s | 1.8 MB 00:00 2024-02-26T06:10:06.356 INFO:teuthology.orchestra.run.smithi006.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 865 kB/s | 42 kB 00:00 2024-02-26T06:10:07.665 INFO:teuthology.orchestra.run.smithi006.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 189 kB/s | 247 kB 00:01 2024-02-26T06:10:07.773 INFO:teuthology.orchestra.run.smithi006.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 664 kB/s | 71 kB 00:00 2024-02-26T06:10:08.132 INFO:teuthology.orchestra.run.smithi006.stdout:(156/166): librados2-19.0.0-1514.g25296a8f.el9. 1.0 MB/s | 3.4 MB 00:03 2024-02-26T06:10:08.365 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-02-26T06:10:08.476 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T06:10:08.522 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T06:10:08.576 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T06:10:08.611 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T06:10:08.636 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T06:10:08.677 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T06:10:08.757 INFO:teuthology.orchestra.run.smithi006.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 37 kB/s | 36 kB 00:00 2024-02-26T06:10:09.003 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T06:10:09.089 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:10:09.227 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:10:09.257 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:10:09.266 INFO:teuthology.orchestra.run.smithi006.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 507 kB/s | 256 kB 00:00 2024-02-26T06:10:09.316 INFO:teuthology.orchestra.run.smithi006.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 471 kB/s | 23 kB 00:00 2024-02-26T06:10:09.347 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:10:09.378 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:10:09.417 INFO:teuthology.orchestra.run.smithi006.stdout:(160/166): elfutils-libelf-0.190-2.el9.x86_64.r 150 kB/s | 192 kB 00:01 2024-02-26T06:10:09.457 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T06:10:09.563 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T06:10:09.744 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T06:10:09.796 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T06:10:09.841 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:10:09.928 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:10:09.960 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:10:10.014 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:10:10.036 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:10:10.054 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:10:10.220 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:10:10.252 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:10:10.354 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T06:10:10.536 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:10:10.596 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:10:10.668 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:10:10.731 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:10:10.940 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:10:11.052 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T06:10:11.146 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T06:10:11.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T06:10:11.230 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T06:10:11.303 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T06:10:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T06:10:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T06:10:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T06:10:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T06:10:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T06:10:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T06:10:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T06:10:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T06:10:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T06:10:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T06:10:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T06:10:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T06:10:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T06:10:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T06:10:12.063 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T06:10:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T06:10:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T06:10:12.339 INFO:teuthology.orchestra.run.smithi006.stdout:(161/166): openssl-3.0.7-25.el9.x86_64.rpm 411 kB/s | 1.2 MB 00:03 2024-02-26T06:10:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T06:10:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T06:10:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T06:10:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T06:10:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T06:10:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T06:10:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T06:10:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T06:10:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T06:10:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T06:10:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T06:10:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T06:10:13.391 INFO:teuthology.orchestra.run.smithi006.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 308 kB/s | 323 kB 00:01 2024-02-26T06:10:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T06:10:13.482 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T06:10:13.519 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T06:10:13.561 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T06:10:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T06:10:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T06:10:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T06:10:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T06:10:13.834 INFO:teuthology.orchestra.run.smithi006.stdout:(163/166): NetworkManager-1.45.91-1.el9.x86_64. 261 kB/s | 2.3 MB 00:08 2024-02-26T06:10:13.867 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T06:10:13.967 INFO:teuthology.orchestra.run.smithi006.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 561 kB/s | 74 kB 00:00 2024-02-26T06:10:13.968 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T06:10:14.009 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T06:10:14.049 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T06:10:14.076 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T06:10:14.101 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T06:10:14.172 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T06:10:14.301 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T06:10:14.343 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T06:10:14.387 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T06:10:14.421 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T06:10:14.460 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T06:10:14.492 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T06:10:14.530 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T06:10:14.565 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T06:10:14.600 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T06:10:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T06:10:14.676 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T06:10:14.763 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T06:10:14.807 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T06:10:14.838 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T06:10:14.876 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T06:10:14.983 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T06:10:15.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T06:10:15.757 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T06:10:15.945 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T06:10:16.141 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T06:10:16.217 INFO:teuthology.orchestra.run.smithi006.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 389 kB/s | 1.1 MB 00:02 2024-02-26T06:10:16.358 INFO:teuthology.orchestra.run.smithi006.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 318 kB/s | 2.2 MB 00:06 2024-02-26T06:10:16.365 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:10:16.365 INFO:teuthology.orchestra.run.smithi006.stdout:Total 8.6 MB/s | 221 MB 00:25 2024-02-26T06:10:16.386 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T06:10:16.481 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T06:10:16.560 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T06:10:16.602 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T06:10:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T06:10:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T06:10:16.894 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T06:10:17.536 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T06:10:17.692 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T06:10:18.920 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T06:10:19.004 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T06:10:19.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T06:10:19.073 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T06:10:19.380 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T06:10:19.412 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:10:19.502 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:10:19.533 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T06:10:19.578 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T06:10:19.945 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-02-26T06:10:20.034 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T06:10:20.091 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:10:20.162 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:10:20.194 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T06:10:20.339 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-02-26T06:10:20.339 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-02-26T06:10:22.394 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:10:22.458 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:10:22.925 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-02-26T06:10:22.926 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-02-26T06:10:22.993 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:10:23.047 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:10:23.128 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:10:23.128 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T06:10:23.128 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T06:10:23.128 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:23.165 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:10:29.995 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-02-26T06:10:30.134 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T06:10:30.242 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T06:10:30.367 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T06:10:30.424 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T06:10:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T06:10:30.562 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T06:10:30.981 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T06:10:31.025 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:10:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:10:31.235 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:10:31.388 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:10:31.425 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:10:31.591 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T06:10:31.736 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T06:10:32.052 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T06:10:32.109 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T06:10:32.134 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:10:32.297 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:10:32.326 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:10:32.420 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:10:32.476 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:10:32.548 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:10:32.838 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:10:32.889 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:10:33.214 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T06:10:33.443 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:10:33.522 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:10:33.662 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:10:33.771 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:10:34.117 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:10:34.299 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T06:10:34.432 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T06:10:34.493 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T06:10:34.523 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T06:10:34.624 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T06:10:34.664 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T06:10:34.730 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T06:10:34.770 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T06:10:34.812 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T06:10:34.934 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T06:10:35.018 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T06:10:35.054 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T06:10:35.119 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T06:10:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T06:10:35.293 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T06:10:35.345 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T06:10:35.383 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T06:10:35.509 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T06:10:35.573 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T06:10:35.686 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T06:10:35.883 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T06:10:35.968 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T06:10:36.142 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-02-26T06:10:36.143 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:36.311 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:10:36.327 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:10:36.327 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:36.327 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T06:10:36.327 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:36.699 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T06:10:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T06:10:36.853 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T06:10:36.906 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T06:10:36.971 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T06:10:37.026 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T06:10:37.102 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T06:10:37.323 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T06:10:37.375 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T06:10:37.458 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T06:10:37.560 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T06:10:37.600 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T06:10:37.641 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T06:10:37.729 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T06:10:37.774 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T06:10:37.814 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:10:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T06:10:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:10:37.840 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:37.840 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T06:10:37.840 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:10:37.840 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:10:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:37.904 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T06:10:38.057 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:10:38.080 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:10:38.080 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:38.080 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T06:10:38.080 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:10:38.081 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:10:38.081 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:38.110 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T06:10:38.162 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T06:10:38.200 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T06:10:38.284 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T06:10:38.430 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T06:10:38.474 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T06:10:38.523 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T06:10:38.525 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:10:38.548 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:10:38.548 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:38.548 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T06:10:38.548 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:10:38.548 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:10:38.548 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:38.557 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T06:10:38.590 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T06:10:38.656 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T06:10:38.713 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T06:10:38.764 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T06:10:38.769 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T06:10:38.962 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T06:10:38.963 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T06:10:39.011 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T06:10:39.011 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T06:10:39.049 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T06:10:39.081 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T06:10:39.094 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T06:10:39.157 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T06:10:39.187 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T06:10:39.209 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T06:10:39.250 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T06:10:39.303 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T06:10:39.325 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T06:10:39.355 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T06:10:39.371 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T06:10:39.412 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T06:10:39.444 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T06:10:39.460 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T06:10:39.480 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T06:10:39.503 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T06:10:39.508 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T06:10:39.653 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T06:10:39.772 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T06:10:39.971 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:10:40.000 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:10:40.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T06:10:40.108 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T06:10:40.140 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:10:40.163 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T06:10:40.261 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T06:10:40.370 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:10:40.399 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:10:40.429 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T06:10:40.489 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T06:10:41.359 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:10:41.450 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:10:41.519 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T06:10:41.549 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:10:41.588 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T06:10:41.622 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T06:10:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T06:10:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:10:41.755 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T06:10:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:10:41.767 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:41.767 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T06:10:41.767 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:10:41.767 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:10:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:41.818 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T06:10:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:10:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:10:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T06:10:41.968 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T06:10:42.014 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T06:10:42.261 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T06:10:42.295 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T06:10:42.393 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T06:10:43.302 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T06:10:43.437 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T06:10:45.111 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T06:10:45.210 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T06:10:45.237 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T06:10:45.263 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T06:10:45.637 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T06:10:45.672 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:10:45.802 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:10:45.837 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T06:10:45.894 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T06:10:46.298 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T06:10:46.400 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T06:10:46.456 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T06:10:46.516 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:10:46.601 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:10:46.627 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T06:10:47.254 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T06:10:47.313 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:10:47.343 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:10:47.343 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:47.343 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T06:10:47.343 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:10:47.344 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:10:47.344 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:47.390 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:10:47.412 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:10:47.412 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:47.412 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T06:10:47.412 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:47.466 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:10:47.488 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:10:47.488 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:47.488 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T06:10:47.489 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:10:47.489 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:10:47.489 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:47.792 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:10:47.814 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:10:47.815 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:10:47.815 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T06:10:47.815 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:10:47.815 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:10:47.815 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:10:47.866 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T06:10:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T06:10:47.940 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T06:10:47.976 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T06:10:48.147 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T06:10:48.200 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T06:10:48.224 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T06:10:48.256 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T06:10:48.294 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T06:10:48.349 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:10:48.402 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T06:10:48.437 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T06:10:49.056 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:10:49.103 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:10:49.130 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:10:49.193 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T06:10:49.262 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:10:49.307 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:10:49.988 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:10:50.006 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T06:10:50.035 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:10:50.133 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T06:10:50.191 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:10:50.192 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T06:10:50.192 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T06:10:50.192 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:10:50.217 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T06:10:50.236 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:10:50.250 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T06:10:50.276 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T06:10:50.301 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T06:10:50.326 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T06:10:50.353 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T06:10:50.353 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:10:50.449 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:10:50.487 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T06:10:50.521 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T06:10:50.548 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T06:10:50.573 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T06:10:50.573 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:10:50.658 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:10:50.683 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T06:10:50.689 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:10:50.689 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:10:50.869 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:10:50.869 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:10:50.961 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:10:50.994 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T06:10:51.021 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T06:10:51.054 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T06:10:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:10:57.589 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T06:10:57.590 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T06:10:57.591 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T06:10:57.592 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T06:10:57.593 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T06:10:57.594 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T06:10:57.595 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T06:10:57.596 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T06:10:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T06:10:57.598 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T06:10:57.599 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T06:10:57.600 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T06:10:57.601 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T06:10:57.602 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T06:11:00.700 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T06:11:00.701 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.702 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T06:11:00.703 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: jq-1.6-16.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.704 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T06:11:00.705 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T06:11:00.706 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T06:11:00.707 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T06:11:00.708 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T06:11:00.709 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T06:11:00.710 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:00.711 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-02-26T06:11:00.712 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-02-26T06:11:01.171 DEBUG:teuthology.parallel:result is None 2024-02-26T06:11:03.600 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:11:03.600 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2024-02-26T06:11:03.600 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2024-02-26T06:11:03.600 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2024-02-26T06:11:03.601 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2024-02-26T06:11:03.601 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2024-02-26T06:11:03.601 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2024-02-26T06:11:03.601 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2024-02-26T06:11:03.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:03.809 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:11:03.853 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:11:03.853 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:03.853 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T06:11:03.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:11:05.600 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:11:05.601 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:05.601 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T06:11:05.601 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:11:05.601 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:11:05.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:05.851 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:11:05.940 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:11:05.940 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:05.940 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T06:11:05.941 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:11:05.941 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:11:05.941 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:11:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:11:06.544 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:06.544 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T06:11:06.544 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:11:06.544 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:11:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T06:11:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T06:11:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T06:11:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T06:11:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T06:11:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T06:11:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T06:11:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T06:11:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T06:11:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T06:11:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T06:11:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T06:11:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T06:11:08.493 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:11:08.535 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:11:08.602 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T06:11:08.719 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T06:11:08.744 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:11:09.092 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:11:09.129 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:11:10.311 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:11:10.337 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:11:10.486 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:11:10.666 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T06:11:10.697 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:11:10.794 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:11:10.794 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:10.794 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T06:11:10.794 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:11:10.794 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:11:10.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:10.848 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:11:10.887 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:11:11.033 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T06:11:16.060 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T06:11:16.143 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T06:11:17.113 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T06:11:17.145 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:11:17.242 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:11:17.242 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:17.242 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T06:11:17.242 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:11:17.242 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:11:17.242 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:17.289 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:11:17.388 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:11:17.389 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:17.389 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T06:11:17.389 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:17.487 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:11:17.594 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:11:17.595 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:17.595 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T06:11:17.595 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:11:17.595 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:11:17.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:11:18.072 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:11:18.072 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:11:18.072 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T06:11:18.072 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:11:18.072 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:11:18.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:18.138 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T06:11:18.197 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T06:11:18.233 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T06:11:18.272 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T06:11:18.588 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T06:11:18.654 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T06:11:18.687 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T06:11:18.726 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T06:11:18.774 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T06:11:18.858 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:11:18.966 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T06:11:18.997 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T06:11:19.840 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:11:19.893 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:11:19.994 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:11:20.064 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T06:11:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T06:11:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T06:11:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T06:11:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T06:11:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T06:11:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T06:11:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T06:11:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T06:11:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:11:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:11:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T06:11:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T06:11:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T06:11:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T06:11:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:11:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:11:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T06:11:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:11:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:11:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:11:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:11:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:11:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T06:11:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T06:11:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T06:11:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:11:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:11:23.742 INFO:teuthology.orchestra.run.smithi006.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-26T06:11:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:11:28.825 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T06:11:28.826 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T06:11:28.827 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T06:11:28.828 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T06:11:28.829 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T06:11:28.830 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T06:11:28.831 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T06:11:28.832 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T06:11:28.833 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T06:11:28.834 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T06:11:28.835 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T06:11:28.836 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T06:11:28.837 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T06:11:31.191 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T06:11:31.192 INFO:teuthology.orchestra.run.smithi006.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.193 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.194 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T06:11:31.195 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-16.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T06:11:31.196 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T06:11:31.197 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.198 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T06:11:31.199 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T06:11:31.200 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T06:11:31.201 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T06:11:31.202 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T06:11:31.203 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T06:11:31.204 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T06:11:31.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:11:31.204 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-02-26T06:11:31.850 DEBUG:teuthology.parallel:result is None 2024-02-26T06:13:37.361 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 80 kB/s | 20 MB 04:22 2024-02-26T06:13:42.351 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-26T06:13:44.926 INFO:teuthology.orchestra.run.smithi167.stderr:Modular dependency problems: 2024-02-26T06:13:44.926 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-02-26T06:13:44.926 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 1: conflicting requests 2024-02-26T06:13:44.926 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.926 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 2: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 3: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 4: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 5: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 6: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 7: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 8: conflicting requests 2024-02-26T06:13:44.927 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 9: conflicting requests 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 10: conflicting requests 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 11: conflicting requests 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 12: conflicting requests 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 13: conflicting requests 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 14: conflicting requests 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.928 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 15: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 16: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 17: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 18: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 19: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 20: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 21: conflicting requests 2024-02-26T06:13:44.929 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 22: conflicting requests 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 23: conflicting requests 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 24: conflicting requests 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 25: conflicting requests 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 26: conflicting requests 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 27: conflicting requests 2024-02-26T06:13:44.930 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 28: conflicting requests 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 29: conflicting requests 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 30: conflicting requests 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 31: conflicting requests 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 32: conflicting requests 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 33: conflicting requests 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.931 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 34: conflicting requests 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 35: conflicting requests 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 36: conflicting requests 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.932 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 38: conflicting requests 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.933 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.934 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T06:13:44.935 INFO:teuthology.orchestra.run.smithi167.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T06:13:45.330 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:13:45.331 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T06:13:45.346 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T06:13:45.348 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T06:13:45.349 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T06:13:45.350 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T06:13:45.352 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T06:13:45.355 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T06:13:45.356 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T06:13:45.447 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-02-26T06:13:45.469 INFO:teuthology.orchestra.run.smithi167.stdout:========================================================================================= 2024-02-26T06:13:45.469 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-02-26T06:13:45.469 INFO:teuthology.orchestra.run.smithi167.stdout:========================================================================================= 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T06:13:45.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T06:13:45.471 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-02-26T06:13:45.472 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T06:13:45.473 INFO:teuthology.orchestra.run.smithi167.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T06:13:45.474 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T06:13:45.475 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T06:13:45.476 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T06:13:45.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T06:13:45.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T06:13:45.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T06:13:45.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T06:13:45.481 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout:========================================================================================= 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout:Install 150 Packages 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 16 Packages 2024-02-26T06:13:45.482 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:13:45.483 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 221 M 2024-02-26T06:13:45.483 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-02-26T06:13:46.310 INFO:teuthology.orchestra.run.smithi167.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-02-26T06:13:46.601 INFO:teuthology.orchestra.run.smithi167.stdout:(2/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 10 MB/s | 5.3 MB 00:00 2024-02-26T06:13:46.651 INFO:teuthology.orchestra.run.smithi167.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 2.8 MB/s | 145 kB 00:00 2024-02-26T06:13:46.818 INFO:teuthology.orchestra.run.smithi167.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 13 MB/s | 2.1 MB 00:00 2024-02-26T06:13:46.935 INFO:teuthology.orchestra.run.smithi167.stdout:(5/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 13 MB/s | 1.5 MB 00:00 2024-02-26T06:13:46.969 INFO:teuthology.orchestra.run.smithi167.stdout:(6/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 1.3 MB/s | 886 kB 00:00 2024-02-26T06:13:47.236 INFO:teuthology.orchestra.run.smithi167.stdout:(7/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 16 MB/s | 4.7 MB 00:00 2024-02-26T06:13:47.853 INFO:teuthology.orchestra.run.smithi167.stdout:(8/166): ceph-radosgw-19.0.0-1514.g25296a8f.el9 16 MB/s | 9.7 MB 00:00 2024-02-26T06:13:47.911 INFO:teuthology.orchestra.run.smithi167.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 433 kB/s | 25 kB 00:00 2024-02-26T06:13:49.678 INFO:teuthology.orchestra.run.smithi167.stdout:(10/166): ceph-common-19.0.0-1514.g25296a8f.el9 5.5 MB/s | 20 MB 00:03 2024-02-26T06:13:49.745 INFO:teuthology.orchestra.run.smithi167.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 3.2 MB/s | 217 kB 00:00 2024-02-26T06:13:49.845 INFO:teuthology.orchestra.run.smithi167.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 271 kB/s | 27 kB 00:00 2024-02-26T06:13:49.945 INFO:teuthology.orchestra.run.smithi167.stdout:(13/166): libcephfs-devel-19.0.0-1514.g25296a8f 312 kB/s | 31 kB 00:00 2024-02-26T06:13:50.146 INFO:teuthology.orchestra.run.smithi167.stdout:(14/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 3.6 MB/s | 743 kB 00:00 2024-02-26T06:13:50.271 INFO:teuthology.orchestra.run.smithi167.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.3 MB/s | 163 kB 00:00 2024-02-26T06:13:50.355 INFO:teuthology.orchestra.run.smithi167.stdout:(16/166): librados-devel-19.0.0-1514.g25296a8f. 1.5 MB/s | 127 kB 00:00 2024-02-26T06:13:50.463 INFO:teuthology.orchestra.run.smithi167.stdout:(17/166): libradosstriper1-19.0.0-1514.g25296a8 4.5 MB/s | 498 kB 00:00 2024-02-26T06:13:50.680 INFO:teuthology.orchestra.run.smithi167.stdout:(18/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x8 4.5 MB/s | 17 MB 00:03 2024-02-26T06:13:51.122 INFO:teuthology.orchestra.run.smithi167.stdout:(19/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 15 MB/s | 47 MB 00:03 2024-02-26T06:13:51.150 INFO:teuthology.orchestra.run.smithi167.stdout:(20/166): python3-ceph-argparse-19.0.0-1514.g25 97 kB/s | 45 kB 00:00 2024-02-26T06:13:51.181 INFO:teuthology.orchestra.run.smithi167.stdout:(21/166): python3-ceph-common-19.0.0-1514.g2529 2.2 MB/s | 131 kB 00:00 2024-02-26T06:13:51.224 INFO:teuthology.orchestra.run.smithi167.stdout:(22/166): python3-cephfs-19.0.0-1514.g25296a8f. 2.1 MB/s | 161 kB 00:00 2024-02-26T06:13:51.249 INFO:teuthology.orchestra.run.smithi167.stdout:(23/166): python3-rados-19.0.0-1514.g25296a8f.e 4.7 MB/s | 320 kB 00:00 2024-02-26T06:13:51.329 INFO:teuthology.orchestra.run.smithi167.stdout:(24/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 5.9 MB/s | 5.1 MB 00:00 2024-02-26T06:13:51.350 INFO:teuthology.orchestra.run.smithi167.stdout:(25/166): python3-rgw-19.0.0-1514.g25296a8f.el9 992 kB/s | 99 kB 00:00 2024-02-26T06:13:51.383 INFO:teuthology.orchestra.run.smithi167.stdout:(26/166): python3-rbd-19.0.0-1514.g25296a8f.el9 1.8 MB/s | 298 kB 00:00 2024-02-26T06:13:51.408 INFO:teuthology.orchestra.run.smithi167.stdout:(27/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-26T06:13:51.473 INFO:teuthology.orchestra.run.smithi167.stdout:(28/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 2.0 MB/s | 171 kB 00:00 2024-02-26T06:13:51.492 INFO:teuthology.orchestra.run.smithi167.stdout:(29/166): ceph-grafana-dashboards-19.0.0-1514.g 375 kB/s | 31 kB 00:00 2024-02-26T06:13:51.548 INFO:teuthology.orchestra.run.smithi167.stdout:(30/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 16 MB/s | 3.1 MB 00:00 2024-02-26T06:13:51.567 INFO:teuthology.orchestra.run.smithi167.stdout:(31/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 1.3 MB/s | 138 kB 00:00 2024-02-26T06:13:51.663 INFO:teuthology.orchestra.run.smithi167.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 2.6 MB/s | 247 kB 00:00 2024-02-26T06:13:51.735 INFO:teuthology.orchestra.run.smithi167.stdout:(33/166): ceph-mgr-dashboard-19.0.0-1514.g25296 7.3 MB/s | 1.8 MB 00:00 2024-02-26T06:13:51.760 INFO:teuthology.orchestra.run.smithi167.stdout:(34/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 521 kB/s | 52 kB 00:00 2024-02-26T06:13:51.819 INFO:teuthology.orchestra.run.smithi167.stdout:(35/166): ceph-prometheus-alerts-19.0.0-1514.g2 183 kB/s | 15 kB 00:00 2024-02-26T06:13:51.844 INFO:teuthology.orchestra.run.smithi167.stdout:(36/166): ceph-volume-19.0.0-1514.g25296a8f.el9 3.1 MB/s | 264 kB 00:00 2024-02-26T06:13:51.986 INFO:teuthology.orchestra.run.smithi167.stdout:(37/166): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-02-26T06:13:52.011 INFO:teuthology.orchestra.run.smithi167.stdout:(38/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 109 kB/s | 18 kB 00:00 2024-02-26T06:13:52.036 INFO:teuthology.orchestra.run.smithi167.stdout:(39/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 3.4 MB/s | 748 kB 00:00 2024-02-26T06:13:52.128 INFO:teuthology.orchestra.run.smithi167.stdout:(40/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 124 kB/s | 18 kB 00:00 2024-02-26T06:13:52.154 INFO:teuthology.orchestra.run.smithi167.stdout:(41/166): fuse-2.9.9-15.el9.x86_64.rpm 565 kB/s | 80 kB 00:00 2024-02-26T06:13:52.187 INFO:teuthology.orchestra.run.smithi167.stdout:(42/166): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-02-26T06:13:52.221 INFO:teuthology.orchestra.run.smithi167.stdout:(43/166): libquadmath-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2024-02-26T06:13:52.263 INFO:teuthology.orchestra.run.smithi167.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 218 kB 00:00 2024-02-26T06:13:52.288 INFO:teuthology.orchestra.run.smithi167.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-02-26T06:13:52.339 INFO:teuthology.orchestra.run.smithi167.stdout:(46/166): libgfortran-11.4.1-3.el9.x86_64.rpm 3.7 MB/s | 803 kB 00:00 2024-02-26T06:13:52.372 INFO:teuthology.orchestra.run.smithi167.stdout:(47/166): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-02-26T06:13:52.397 INFO:teuthology.orchestra.run.smithi167.stdout:(48/166): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-02-26T06:13:52.423 INFO:teuthology.orchestra.run.smithi167.stdout:(49/166): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-02-26T06:13:52.465 INFO:teuthology.orchestra.run.smithi167.stdout:(50/166): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-02-26T06:13:52.490 INFO:teuthology.orchestra.run.smithi167.stdout:(51/166): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-02-26T06:13:52.557 INFO:teuthology.orchestra.run.smithi167.stdout:(52/166): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 689 kB 00:00 2024-02-26T06:13:52.582 INFO:teuthology.orchestra.run.smithi167.stdout:(53/166): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-02-26T06:13:52.640 INFO:teuthology.orchestra.run.smithi167.stdout:(54/166): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-02-26T06:13:52.665 INFO:teuthology.orchestra.run.smithi167.stdout:(55/166): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-02-26T06:13:52.707 INFO:teuthology.orchestra.run.smithi167.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-02-26T06:13:52.732 INFO:teuthology.orchestra.run.smithi167.stdout:(57/166): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-02-26T06:13:52.766 INFO:teuthology.orchestra.run.smithi167.stdout:(58/166): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-02-26T06:13:52.808 INFO:teuthology.orchestra.run.smithi167.stdout:(59/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-02-26T06:13:52.825 INFO:teuthology.orchestra.run.smithi167.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-02-26T06:13:52.884 INFO:teuthology.orchestra.run.smithi167.stdout:(61/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-02-26T06:13:52.934 INFO:teuthology.orchestra.run.smithi167.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 513 kB/s | 26 kB 00:00 2024-02-26T06:13:52.959 INFO:teuthology.orchestra.run.smithi167.stdout:(63/166): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-02-26T06:13:52.993 INFO:teuthology.orchestra.run.smithi167.stdout:(64/166): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-02-26T06:13:53.018 INFO:teuthology.orchestra.run.smithi167.stdout:(65/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-02-26T06:13:53.068 INFO:teuthology.orchestra.run.smithi167.stdout:(66/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-02-26T06:13:53.094 INFO:teuthology.orchestra.run.smithi167.stdout:(67/166): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-02-26T06:13:53.127 INFO:teuthology.orchestra.run.smithi167.stdout:(68/166): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-02-26T06:13:53.319 INFO:teuthology.orchestra.run.smithi167.stdout:(69/166): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-02-26T06:13:53.361 INFO:teuthology.orchestra.run.smithi167.stdout:(70/166): postgresql-13.11-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-02-26T06:13:53.420 INFO:teuthology.orchestra.run.smithi167.stdout:(71/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 13 MB/s | 4.1 MB 00:00 2024-02-26T06:13:53.479 INFO:teuthology.orchestra.run.smithi167.stdout:(72/166): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 135 kB 00:00 2024-02-26T06:13:53.521 INFO:teuthology.orchestra.run.smithi167.stdout:(73/166): postgresql-contrib-13.11-1.el9.x86_64 4.4 MB/s | 905 kB 00:00 2024-02-26T06:13:53.579 INFO:teuthology.orchestra.run.smithi167.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-02-26T06:13:53.646 INFO:teuthology.orchestra.run.smithi167.stdout:(75/166): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-02-26T06:13:53.738 INFO:teuthology.orchestra.run.smithi167.stdout:(76/166): postgresql-server-13.11-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2024-02-26T06:13:53.780 INFO:teuthology.orchestra.run.smithi167.stdout:(77/166): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-02-26T06:13:53.822 INFO:teuthology.orchestra.run.smithi167.stdout:(78/166): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-02-26T06:13:53.839 INFO:teuthology.orchestra.run.smithi167.stdout:(79/166): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-02-26T06:13:53.881 INFO:teuthology.orchestra.run.smithi167.stdout:(80/166): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-02-26T06:13:53.923 INFO:teuthology.orchestra.run.smithi167.stdout:(81/166): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-02-26T06:13:53.949 INFO:teuthology.orchestra.run.smithi167.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-02-26T06:13:54.024 INFO:teuthology.orchestra.run.smithi167.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-02-26T06:13:54.133 INFO:teuthology.orchestra.run.smithi167.stdout:(84/166): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-02-26T06:13:54.174 INFO:teuthology.orchestra.run.smithi167.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-02-26T06:13:54.208 INFO:teuthology.orchestra.run.smithi167.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-02-26T06:13:54.233 INFO:teuthology.orchestra.run.smithi167.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-02-26T06:13:54.301 INFO:teuthology.orchestra.run.smithi167.stdout:(88/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-02-26T06:13:54.342 INFO:teuthology.orchestra.run.smithi167.stdout:(89/166): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-02-26T06:13:54.368 INFO:teuthology.orchestra.run.smithi167.stdout:(90/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-02-26T06:13:54.393 INFO:teuthology.orchestra.run.smithi167.stdout:(91/166): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-02-26T06:13:54.418 INFO:teuthology.orchestra.run.smithi167.stdout:(92/166): python3-pytz-2021.1-5.el9.noarch.rpm 673 kB/s | 51 kB 00:00 2024-02-26T06:13:54.443 INFO:teuthology.orchestra.run.smithi167.stdout:(93/166): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-02-26T06:13:54.494 INFO:teuthology.orchestra.run.smithi167.stdout:(94/166): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-02-26T06:13:54.519 INFO:teuthology.orchestra.run.smithi167.stdout:(95/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 585 kB/s | 44 kB 00:00 2024-02-26T06:13:54.570 INFO:teuthology.orchestra.run.smithi167.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 698 kB/s | 35 kB 00:00 2024-02-26T06:13:54.595 INFO:teuthology.orchestra.run.smithi167.stdout:(97/166): qatlib-23.11.0-1.el9.x86_64.rpm 2.1 MB/s | 216 kB 00:00 2024-02-26T06:13:54.645 INFO:teuthology.orchestra.run.smithi167.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 592 kB/s | 44 kB 00:00 2024-02-26T06:13:54.696 INFO:teuthology.orchestra.run.smithi167.stdout:(99/166): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-02-26T06:13:54.721 INFO:teuthology.orchestra.run.smithi167.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 764 kB/s | 58 kB 00:00 2024-02-26T06:13:54.780 INFO:teuthology.orchestra.run.smithi167.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-02-26T06:13:54.805 INFO:teuthology.orchestra.run.smithi167.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-02-26T06:13:54.872 INFO:teuthology.orchestra.run.smithi167.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-02-26T06:13:54.897 INFO:teuthology.orchestra.run.smithi167.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-02-26T06:13:55.064 INFO:teuthology.orchestra.run.smithi167.stdout:(105/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-02-26T06:13:55.106 INFO:teuthology.orchestra.run.smithi167.stdout:(106/166): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-02-26T06:13:55.382 INFO:teuthology.orchestra.run.smithi167.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-02-26T06:13:55.573 INFO:teuthology.orchestra.run.smithi167.stdout:(108/166): abseil-cpp-20211102.0-3.el9.x86_64.r 811 kB/s | 548 kB 00:00 2024-02-26T06:13:55.599 INFO:teuthology.orchestra.run.smithi167.stdout:(109/166): gperftools-libs-2.9.1-2.el9.x86_64.r 578 kB/s | 309 kB 00:00 2024-02-26T06:13:55.666 INFO:teuthology.orchestra.run.smithi167.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-02-26T06:13:55.699 INFO:teuthology.orchestra.run.smithi167.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-02-26T06:13:55.766 INFO:teuthology.orchestra.run.smithi167.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-02-26T06:13:55.817 INFO:teuthology.orchestra.run.smithi167.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-02-26T06:13:55.959 INFO:teuthology.orchestra.run.smithi167.stdout:(114/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-02-26T06:13:56.009 INFO:teuthology.orchestra.run.smithi167.stdout:(115/166): python3-asyncssh-2.13.2-4.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-02-26T06:13:56.068 INFO:teuthology.orchestra.run.smithi167.stdout:(116/166): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-02-26T06:13:56.084 INFO:teuthology.orchestra.run.smithi167.stdout:(117/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-02-26T06:13:56.110 INFO:teuthology.orchestra.run.smithi167.stdout:(118/166): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-02-26T06:13:56.168 INFO:teuthology.orchestra.run.smithi167.stdout:(119/166): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-02-26T06:13:56.194 INFO:teuthology.orchestra.run.smithi167.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-02-26T06:13:56.228 INFO:teuthology.orchestra.run.smithi167.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-02-26T06:13:56.278 INFO:teuthology.orchestra.run.smithi167.stdout:(122/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-02-26T06:13:56.303 INFO:teuthology.orchestra.run.smithi167.stdout:(123/166): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-02-26T06:13:56.329 INFO:teuthology.orchestra.run.smithi167.stdout:(124/166): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-02-26T06:13:56.429 INFO:teuthology.orchestra.run.smithi167.stdout:(125/166): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-02-26T06:13:56.454 INFO:teuthology.orchestra.run.smithi167.stdout:(126/166): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-02-26T06:13:56.480 INFO:teuthology.orchestra.run.smithi167.stdout:(127/166): python3-grpcio-tools-1.46.7-10.el9.x 819 kB/s | 144 kB 00:00 2024-02-26T06:13:56.522 INFO:teuthology.orchestra.run.smithi167.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 191 kB/s | 18 kB 00:00 2024-02-26T06:13:56.547 INFO:teuthology.orchestra.run.smithi167.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-02-26T06:13:56.580 INFO:teuthology.orchestra.run.smithi167.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-02-26T06:13:56.614 INFO:teuthology.orchestra.run.smithi167.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-02-26T06:13:56.664 INFO:teuthology.orchestra.run.smithi167.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-02-26T06:13:56.690 INFO:teuthology.orchestra.run.smithi167.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 424 kB/s | 46 kB 00:00 2024-02-26T06:13:56.715 INFO:teuthology.orchestra.run.smithi167.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-02-26T06:13:56.757 INFO:teuthology.orchestra.run.smithi167.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-02-26T06:13:56.790 INFO:teuthology.orchestra.run.smithi167.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-02-26T06:13:56.816 INFO:teuthology.orchestra.run.smithi167.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-02-26T06:13:56.849 INFO:teuthology.orchestra.run.smithi167.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 973 kB/s | 90 kB 00:00 2024-02-26T06:13:56.883 INFO:teuthology.orchestra.run.smithi167.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-02-26T06:13:56.908 INFO:teuthology.orchestra.run.smithi167.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-02-26T06:13:56.942 INFO:teuthology.orchestra.run.smithi167.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-02-26T06:13:56.976 INFO:teuthology.orchestra.run.smithi167.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-02-26T06:13:57.001 INFO:teuthology.orchestra.run.smithi167.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-02-26T06:13:57.034 INFO:teuthology.orchestra.run.smithi167.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-02-26T06:13:57.077 INFO:teuthology.orchestra.run.smithi167.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-26T06:13:57.102 INFO:teuthology.orchestra.run.smithi167.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 890 kB/s | 90 kB 00:00 2024-02-26T06:13:57.144 INFO:teuthology.orchestra.run.smithi167.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-02-26T06:13:57.177 INFO:teuthology.orchestra.run.smithi167.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-02-26T06:13:57.203 INFO:teuthology.orchestra.run.smithi167.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-26T06:13:57.286 INFO:teuthology.orchestra.run.smithi167.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-02-26T06:13:57.470 INFO:teuthology.orchestra.run.smithi167.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-02-26T06:13:57.529 INFO:teuthology.orchestra.run.smithi167.stdout:(152/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 9.6 MB/s | 3.1 MB 00:00 2024-02-26T06:13:57.579 INFO:teuthology.orchestra.run.smithi167.stdout:(153/166): NetworkManager-libnm-1.45.91-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-02-26T06:13:57.604 INFO:teuthology.orchestra.run.smithi167.stdout:(154/166): NetworkManager-team-1.45.91-1.el9.x8 564 kB/s | 42 kB 00:00 2024-02-26T06:13:57.630 INFO:teuthology.orchestra.run.smithi167.stdout:(155/166): NetworkManager-tui-1.45.91-1.el9.x86 4.8 MB/s | 247 kB 00:00 2024-02-26T06:13:57.663 INFO:teuthology.orchestra.run.smithi167.stdout:(156/166): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-02-26T06:13:57.689 INFO:teuthology.orchestra.run.smithi167.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 624 kB/s | 36 kB 00:00 2024-02-26T06:13:57.739 INFO:teuthology.orchestra.run.smithi167.stdout:(158/166): librados2-19.0.0-1514.g25296a8f.el9. 6.0 MB/s | 3.4 MB 00:00 2024-02-26T06:13:57.773 INFO:teuthology.orchestra.run.smithi167.stdout:(159/166): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-02-26T06:13:57.806 INFO:teuthology.orchestra.run.smithi167.stdout:(160/166): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-02-26T06:13:57.831 INFO:teuthology.orchestra.run.smithi167.stdout:(161/166): libacl-2.3.1-4.el9.x86_64.rpm 250 kB/s | 23 kB 00:00 2024-02-26T06:13:57.898 INFO:teuthology.orchestra.run.smithi167.stdout:(162/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 23 MB/s | 2.2 MB 00:00 2024-02-26T06:13:57.933 INFO:teuthology.orchestra.run.smithi167.stdout:(163/166): openssl-3.0.7-25.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-02-26T06:13:57.957 INFO:teuthology.orchestra.run.smithi167.stdout:(164/166): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-02-26T06:13:57.991 INFO:teuthology.orchestra.run.smithi167.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-26T06:13:58.025 INFO:teuthology.orchestra.run.smithi167.stdout:(166/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 810 kB/s | 74 kB 00:00 2024-02-26T06:13:58.029 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:13:58.029 INFO:teuthology.orchestra.run.smithi167.stdout:Total 18 MB/s | 221 MB 00:12 2024-02-26T06:13:59.258 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-02-26T06:13:59.518 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-02-26T06:13:59.519 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-02-26T06:14:01.318 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-02-26T06:14:01.318 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-02-26T06:14:07.876 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-02-26T06:14:07.987 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T06:14:08.025 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T06:14:08.078 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T06:14:08.113 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T06:14:08.138 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T06:14:08.185 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T06:14:08.507 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T06:14:08.565 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:14:08.686 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:14:08.715 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:14:08.807 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:14:08.836 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:14:08.910 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T06:14:08.996 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T06:14:09.164 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T06:14:09.212 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T06:14:09.240 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:14:09.343 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T06:14:09.366 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:14:09.428 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T06:14:09.450 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:14:09.466 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T06:14:09.659 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:14:09.696 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T06:14:09.817 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T06:14:09.924 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:14:09.984 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T06:14:10.046 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:14:10.152 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:14:10.379 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T06:14:10.496 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T06:14:10.589 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T06:14:10.625 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T06:14:10.659 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T06:14:10.725 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T06:14:10.758 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T06:14:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T06:14:10.843 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T06:14:10.871 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T06:14:10.940 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T06:14:10.994 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T06:14:11.024 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T06:14:11.059 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T06:14:11.136 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T06:14:11.179 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T06:14:11.230 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T06:14:11.259 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T06:14:11.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T06:14:11.373 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T06:14:11.447 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T06:14:11.590 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T06:14:11.651 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T06:14:12.176 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T06:14:12.264 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T06:14:12.300 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T06:14:12.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T06:14:12.370 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T06:14:12.407 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T06:14:12.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T06:14:12.575 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T06:14:12.628 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T06:14:12.682 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T06:14:12.764 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T06:14:12.803 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T06:14:12.840 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T06:14:12.885 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T06:14:12.922 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T06:14:12.965 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T06:14:13.006 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T06:14:13.151 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T06:14:13.204 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T06:14:13.234 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T06:14:13.276 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T06:14:13.387 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T06:14:13.426 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T06:14:13.466 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T06:14:13.492 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T06:14:13.516 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T06:14:13.598 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T06:14:13.727 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T06:14:13.774 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T06:14:13.844 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T06:14:14.071 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T06:14:14.111 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T06:14:14.151 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T06:14:14.180 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T06:14:14.215 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T06:14:14.248 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T06:14:14.299 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T06:14:14.358 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T06:14:14.451 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T06:14:14.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T06:14:14.535 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T06:14:14.583 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T06:14:14.690 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T06:14:14.754 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T06:14:15.490 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T06:14:15.534 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T06:14:15.568 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T06:14:15.668 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T06:14:15.735 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T06:14:15.832 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T06:14:15.879 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T06:14:16.057 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T06:14:16.085 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T06:14:16.163 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T06:14:16.815 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T06:14:16.951 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T06:14:18.177 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T06:14:18.266 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T06:14:18.293 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T06:14:18.327 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T06:14:18.634 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T06:14:18.665 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:14:18.756 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T06:14:18.787 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T06:14:18.831 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T06:14:19.294 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T06:14:19.319 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:14:19.373 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T06:14:19.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T06:14:21.518 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:14:21.553 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:14:22.092 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T06:14:22.140 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:14:22.222 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T06:14:22.222 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T06:14:22.222 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T06:14:22.222 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-02-26T06:14:34.142 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:34.315 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:14:34.330 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T06:14:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T06:14:34.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:35.819 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:14:35.844 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T06:14:35.844 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:35.844 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T06:14:35.844 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:14:35.844 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T06:14:35.845 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:36.061 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:14:36.083 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T06:14:36.084 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:36.084 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T06:14:36.084 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:14:36.084 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T06:14:36.084 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:36.520 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:14:36.542 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T06:14:36.542 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:36.542 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T06:14:36.542 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:14:36.542 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T06:14:36.542 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:36.650 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T06:14:36.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T06:14:36.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T06:14:36.948 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T06:14:37.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T06:14:37.083 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T06:14:37.179 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T06:14:37.311 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T06:14:37.355 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T06:14:37.388 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T06:14:37.419 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T06:14:37.455 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T06:14:37.483 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T06:14:37.851 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:14:37.882 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T06:14:37.913 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T06:14:37.974 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T06:14:38.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:14:38.238 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T06:14:38.264 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:14:39.228 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T06:14:39.256 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:14:39.358 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T06:14:39.552 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T06:14:39.584 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:14:39.607 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T06:14:39.607 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:39.607 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T06:14:39.607 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:14:39.607 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T06:14:39.607 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:39.651 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:14:39.663 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T06:14:39.760 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T06:14:44.138 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T06:14:44.204 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T06:14:45.055 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T06:14:45.084 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:14:45.106 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T06:14:45.106 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:45.106 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T06:14:45.106 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:14:45.106 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T06:14:45.107 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:45.151 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:14:45.173 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T06:14:45.173 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:45.173 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T06:14:45.173 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:45.218 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:14:45.241 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T06:14:45.241 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:45.241 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T06:14:45.241 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:14:45.241 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T06:14:45.241 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:45.536 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:14:45.558 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T06:14:45.559 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T06:14:45.559 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T06:14:45.559 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:14:45.559 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T06:14:45.559 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:45.619 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T06:14:45.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T06:14:45.701 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T06:14:45.728 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T06:14:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T06:14:45.959 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T06:14:45.991 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T06:14:46.029 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T06:14:46.061 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T06:14:46.127 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:14:46.183 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T06:14:46.219 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T06:14:46.782 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:14:46.817 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:14:46.844 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T06:14:46.898 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T06:14:47.744 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T06:14:47.839 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T06:14:47.919 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T06:14:47.960 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T06:14:47.985 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T06:14:48.010 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T06:14:48.036 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T06:14:48.062 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T06:14:48.062 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:14:48.150 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T06:14:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T06:14:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T06:14:48.239 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T06:14:48.264 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T06:14:48.264 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:14:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T06:14:48.374 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T06:14:48.380 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:14:48.380 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:14:48.560 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T06:14:48.560 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:14:48.628 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T06:14:48.660 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T06:14:48.687 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T06:14:48.721 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T06:14:48.802 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:14:53.916 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T06:14:53.916 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T06:14:53.916 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T06:14:53.916 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T06:14:53.916 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T06:14:53.917 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T06:14:53.918 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T06:14:53.919 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T06:14:53.920 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T06:14:53.921 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T06:14:53.922 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T06:14:53.923 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T06:14:53.924 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T06:14:53.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T06:14:53.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T06:14:53.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T06:14:53.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T06:14:53.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T06:14:56.645 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T06:14:56.646 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.6-16.el9.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T06:14:56.649 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T06:14:56.650 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T06:14:56.651 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T06:14:56.652 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T06:14:56.653 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T06:14:56.654 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T06:14:56.655 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.656 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:14:56.657 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-02-26T06:14:57.193 DEBUG:teuthology.parallel:result is None 2024-02-26T06:14:57.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:14:57.337 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T06:14:57.415 INFO:teuthology.orchestra.run.smithi006.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T06:14:57.416 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T06:14:57.416 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T06:14:57.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:14:57.559 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T06:14:57.586 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T06:14:57.587 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T06:14:57.587 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T06:14:57.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:14:57.729 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T06:14:57.753 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T06:14:57.753 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T06:14:57.753 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T06:14:57.755 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-26T06:14:57.755 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:14:57.755 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T06:14:57.871 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:14:57.871 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T06:14:57.905 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:14:57.906 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T06:14:57.939 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-26T06:14:57.939 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:14:57.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T06:14:58.040 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T06:14:58.148 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:14:58.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T06:14:58.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T06:14:58.250 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:14:58.251 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T06:14:58.283 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T06:14:58.354 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-26T06:14:58.354 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:14:58.354 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T06:14:58.456 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T06:14:58.590 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:14:58.590 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T06:14:58.621 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T06:14:58.692 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:14:58.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T06:14:58.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T06:14:58.798 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-26T06:14:58.798 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:14:58.798 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T06:14:58.886 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T06:14:59.017 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:14:59.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T06:14:59.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T06:14:59.119 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:14:59.119 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T06:14:59.151 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T06:14:59.222 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-26T06:14:59.323 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-02-26T06:14:59.324 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:14:59.324 INFO:tasks.cephadm:Cluster fsid is 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:14:59.324 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-26T06:14:59.324 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.6', 'mon.b': '172.21.15.90', 'mon.c': '172.21.15.167'} 2024-02-26T06:14:59.325 INFO:tasks.cephadm:First mon is mon.a on smithi006 2024-02-26T06:14:59.325 INFO:tasks.cephadm:First mgr is x 2024-02-26T06:14:59.325 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-26T06:14:59.325 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname $(hostname -s) 2024-02-26T06:14:59.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-02-26T06:14:59.449 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-02-26T06:14:59.481 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-26T06:14:59.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:14:59.624 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76813/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1514-g25296a8f', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1514.g25296a8f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-16 07:18:11.982893', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240216.060239', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-26T06:14:59.766 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20240216.060239, sha1 25296a8f176256aea5ed8b5850b25167dce0ae33 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:14:59.768 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm 2024-02-26T06:14:59.768 DEBUG:teuthology.orchestra.run.smithi006:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:00.431 INFO:teuthology.orchestra.run.smithi006.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 06:15 /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:00.431 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:00.836 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 06:15 /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:00.836 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:01.420 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 06:15 /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:01.421 DEBUG:teuthology.orchestra.run.smithi006:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:01.485 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:01.504 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T06:15:01.560 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 on all hosts... 2024-02-26T06:15:01.561 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T06:15:01.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T06:15:01.571 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T06:15:02.422 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:15:02.445 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:15:02.838 INFO:teuthology.orchestra.run.smithi006.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:15:40.561 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:15:40.561 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T06:15:40.561 INFO:teuthology.orchestra.run.smithi006.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T06:15:40.561 INFO:teuthology.orchestra.run.smithi006.stdout: "repo_digests": [ 2024-02-26T06:15:40.561 INFO:teuthology.orchestra.run.smithi006.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T06:15:40.562 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:15:40.562 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:15:43.369 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-02-26T06:15:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T06:15:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T06:15:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-02-26T06:15:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T06:15:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-02-26T06:15:43.370 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-02-26T06:15:46.207 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-02-26T06:15:46.232 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph 2024-02-26T06:15:46.341 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-02-26T06:15:46.376 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-02-26T06:15:46.407 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /etc/ceph 2024-02-26T06:15:46.505 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-02-26T06:15:46.536 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-02-26T06:15:46.565 INFO:tasks.cephadm:Writing seed config... 2024-02-26T06:15:46.566 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-26T06:15:46.566 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-26T06:15:46.566 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-26T06:15:46.566 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-02-26T06:15:46.567 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-26T06:15:46.568 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-26T06:15:46.569 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-02-26T06:15:46.570 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:15:46.570 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-26T06:15:46.630 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5e92f702-d46e-11ee-95c3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-26T06:15:46.630 DEBUG:teuthology.orchestra.run.smithi006:mon.a> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service 2024-02-26T06:15:46.677 DEBUG:teuthology.orchestra.run.smithi006:mgr.x> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x.service 2024-02-26T06:15:46.723 INFO:tasks.cephadm:Bootstrapping... 2024-02-26T06:15:46.723 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 -v bootstrap --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.6 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:15:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-02-26T06:15:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33', '-v', 'bootstrap', '--fsid', '5e92f702-d46e-11ee-95c3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.6', '--skip-admin-label'] 2024-02-26T06:15:47.062 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T06:15:47.063 INFO:teuthology.orchestra.run.smithi006.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-26T06:15:47.063 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying podman|docker is present... 2024-02-26T06:15:47.111 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T06:15:47.111 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying lvm2 is present... 2024-02-26T06:15:47.111 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying time synchronization is in place... 2024-02-26T06:15:47.126 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T06:15:47.126 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T06:15:47.140 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T06:15:47.140 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-02-26T06:15:47.156 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-02-26T06:15:47.172 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-02-26T06:15:47.172 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-02-26T06:15:47.172 INFO:teuthology.orchestra.run.smithi006.stdout:Repeating the final host check... 2024-02-26T06:15:47.222 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T06:15:47.222 INFO:teuthology.orchestra.run.smithi006.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-26T06:15:47.223 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl is present 2024-02-26T06:15:47.223 INFO:teuthology.orchestra.run.smithi006.stdout:lvcreate is present 2024-02-26T06:15:47.236 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T06:15:47.236 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T06:15:47.249 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T06:15:47.249 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-02-26T06:15:47.264 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-02-26T06:15:47.278 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-02-26T06:15:47.278 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-02-26T06:15:47.278 INFO:teuthology.orchestra.run.smithi006.stdout:Host looks OK 2024-02-26T06:15:47.278 INFO:teuthology.orchestra.run.smithi006.stdout:Cluster fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:15:47.278 INFO:teuthology.orchestra.run.smithi006.stdout:Acquiring lock 140700130719344 on /run/cephadm/5e92f702-d46e-11ee-95c3-87774f69a715.lock 2024-02-26T06:15:47.279 INFO:teuthology.orchestra.run.smithi006.stdout:Lock 140700130719344 acquired on /run/cephadm/5e92f702-d46e-11ee-95c3-87774f69a715.lock 2024-02-26T06:15:47.279 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 3300 ... 2024-02-26T06:15:47.279 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 6789 ... 2024-02-26T06:15:47.279 INFO:teuthology.orchestra.run.smithi006.stdout:Base mon IP(s) is [172.21.15.6:3300, 172.21.15.6:6789], mon addrv is [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-02-26T06:15:47.287 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.6 metric 100 2024-02-26T06:15:47.287 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.6 metric 100 2024-02-26T06:15:47.294 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-26T06:15:47.294 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-26T06:15:47.294 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-26T06:15:47.301 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-26T06:15:47.301 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-26T06:15:47.301 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T06:15:47.301 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-26T06:15:47.301 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-02-26T06:15:47.301 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T06:15:47.302 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-02-26T06:15:47.302 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-02-26T06:15:47.302 INFO:teuthology.orchestra.run.smithi006.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-26T06:15:47.303 INFO:teuthology.orchestra.run.smithi006.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-26T06:15:47.304 INFO:teuthology.orchestra.run.smithi006.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:15:50.923 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 2024-02-26T06:15:50.923 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T06:15:50.924 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Getting image source signatures 2024-02-26T06:15:50.924 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:3023f2ea73c33e75bbdc7c6f4507678b80f8a5636a9a11c7f009befea7a064c0 2024-02-26T06:15:50.924 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-26T06:15:50.924 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying config sha256:a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 2024-02-26T06:15:50.924 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-26T06:15:52.341 INFO:teuthology.orchestra.run.smithi006.stdout:ceph: stdout ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev) 2024-02-26T06:15:52.341 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph version: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev) 2024-02-26T06:15:52.341 INFO:teuthology.orchestra.run.smithi006.stdout:Extracting ceph user uid/gid from container image... 2024-02-26T06:15:53.511 INFO:teuthology.orchestra.run.smithi006.stdout:stat: stdout 167 167 2024-02-26T06:15:53.511 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial keys... 2024-02-26T06:15:54.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCaLNxla/wlAhAA2YmkMTb5iRPBmgtgBGaFwg== 2024-02-26T06:15:55.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCbLNxl/tT/DRAAzLZjevNkwDDbr/ylcsRo3g== 2024-02-26T06:15:57.958 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCcLNxlJKKLFRAACjIWRiIVMAbDI7TIXQBJOg== 2024-02-26T06:15:57.959 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial monmap... 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool for a [v2:172.21.15.6:3300,v1:172.21.15.6:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = quincy 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: set fsid to 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:15:59.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T06:15:59.103 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:15:59.103 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mon... 2024-02-26T06:16:00.249 INFO:teuthology.orchestra.run.smithi006.stdout:create mon.a on 2024-02-26T06:16:00.642 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-26T06:16:00.874 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-26T06:16:01.169 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5e92f702-d46e-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-5e92f702-d46e-11ee-95c3-87774f69a715.target. 2024-02-26T06:16:01.169 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5e92f702-d46e-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-5e92f702-d46e-11ee-95c3-87774f69a715.target. 2024-02-26T06:16:01.566 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a 2024-02-26T06:16:01.566 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service: Unit ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service not loaded. 2024-02-26T06:16:01.855 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5e92f702-d46e-11ee-95c3-87774f69a715.target.wants/ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-5e92f702-d46e-11ee-95c3-87774f69a715@.service. 2024-02-26T06:16:02.582 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:02 smithi006 podman[31409]: 2024-02-26 06:16:02.50342777 +0000 UTC m=+0.372748284 container start 81ca905c869a1e1bcabceee1b8d8500458bcf4bd1dcd05a4fd22f55e1f3a3554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:16:02.582 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:02 smithi006 bash[31409]: 81ca905c869a1e1bcabceee1b8d8500458bcf4bd1dcd05a4fd22f55e1f3a3554 2024-02-26T06:16:02.614 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:16:02.614 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-02-26T06:16:02.629 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:16:02.629 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-02-26T06:16:02.630 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-02-26T06:16:02.630 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T06:16:02.631 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon to start... 2024-02-26T06:16:02.631 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon... 2024-02-26T06:16:02.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:02 smithi006 ceph-mon[31427]: mkfs 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:02.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:02 smithi006 systemd[1]: Started Ceph mon.a for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:16:02.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:02 smithi006 ceph-mon[31427]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T06:16:03.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:03 smithi006 ceph-mon[31427]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T06:16:03.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:03 smithi006 ceph-mon[31427]: monmap e1: 1 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:16:03.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:03 smithi006 ceph-mon[31427]: fsmap 2024-02-26T06:16:03.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:03 smithi006 ceph-mon[31427]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T06:16:03.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:03 smithi006 ceph-mon[31427]: mgrmap e1: no daemons active 2024-02-26T06:16:03.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:03 smithi006 ceph-mon[31427]: from='client.? 172.21.15.6:0/1711305399' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout cluster: 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout id: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout services: 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.794228s) 2024-02-26T06:16:04.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-26T06:16:04.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-26T06:16:04.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:04.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout data: 2024-02-26T06:16:04.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-26T06:16:04.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pgs: 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.280+0000 7f6f6a200700 1 Processor -- start 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.280+0000 7f6f6a200700 1 -- start start 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.281+0000 7f6f6a200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f64081bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.281+0000 7f6f6a200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f64082180 con 0x7f6f64081790 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.281+0000 7f6f63400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f64081bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.281+0000 7f6f63400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f64081bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51604/0 (socket says 172.21.15.6:51604) 2024-02-26T06:16:04.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.281+0000 7f6f63400700 1 -- 172.21.15.6:0/334113003 learned_addr learned my addr 172.21.15.6:0/334113003 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.282+0000 7f6f63400700 1 -- 172.21.15.6:0/334113003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6410ef60 con 0x7f6f64081790 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.282+0000 7f6f63400700 1 --2- 172.21.15.6:0/334113003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f64081bb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098a0 tx=0x7f6f54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674fed1ec372d888 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.282+0000 7f6f62000700 1 -- 172.21.15.6:0/334113003 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f54004160 con 0x7f6f64081790 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f62000700 1 -- 172.21.15.6:0/334113003 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f540042c0 con 0x7f6f64081790 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f62000700 1 -- 172.21.15.6:0/334113003 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f540045a0 con 0x7f6f64081790 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 -- 172.21.15.6:0/334113003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 msgr2=0x7f6f64081bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 --2- 172.21.15.6:0/334113003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f64081bb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098a0 tx=0x7f6f54009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 -- 172.21.15.6:0/334113003 shutdown_connections 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 --2- 172.21.15.6:0/334113003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f64081bb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 -- 172.21.15.6:0/334113003 >> 172.21.15.6:0/334113003 conn(0x7f6f6407cd50 msgr2=0x7f6f6407f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 -- 172.21.15.6:0/334113003 shutdown_connections 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.283+0000 7f6f6a200700 1 -- 172.21.15.6:0/334113003 wait complete. 2024-02-26T06:16:04.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.284+0000 7f6f6a200700 1 Processor -- start 2024-02-26T06:16:04.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.285+0000 7f6f6a200700 1 -- start start 2024-02-26T06:16:04.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.285+0000 7f6f6a200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f641a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:04.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.285+0000 7f6f6a200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f641a1790 con 0x7f6f64081790 2024-02-26T06:16:04.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.285+0000 7f6f63400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f641a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.286+0000 7f6f63400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f641a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51614/0 (socket says 172.21.15.6:51614) 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.286+0000 7f6f63400700 1 -- 172.21.15.6:0/1711305399 learned_addr learned my addr 172.21.15.6:0/1711305399 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.286+0000 7f6f63400700 1 -- 172.21.15.6:0/1711305399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f54009550 con 0x7f6f64081790 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.286+0000 7f6f63400700 1 --2- 172.21.15.6:0/1711305399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f641a1250 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6f54004000 tx=0x7f6f54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f54011630 con 0x7f6f64081790 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54011790 con 0x7f6f64081790 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f54011a30 con 0x7f6f64081790 2024-02-26T06:16:04.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f641a1990 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f641a1e30 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6f54022070 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.287+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6f5401c450 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.288+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6419aec0 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.291+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6f54011b90 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.373+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6f6410f540 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.374+0000 7f6f5be00700 1 -- 172.21.15.6:0/1711305399 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f6f54011d30 con 0x7f6f64081790 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 msgr2=0x7f6f641a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 --2- 172.21.15.6:0/1711305399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f641a1250 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6f54004000 tx=0x7f6f54004750 comp rx=0 tx=0).stop 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 shutdown_connections 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 --2- 172.21.15.6:0/1711305399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f64081790 0x7f6f641a1250 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:04.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 >> 172.21.15.6:0/1711305399 conn(0x7f6f6407cd50 msgr2=0x7f6f64198000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:04.017 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 shutdown_connections 2024-02-26T06:16:04.017 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:03.375+0000 7f6f6a200700 1 -- 172.21.15.6:0/1711305399 wait complete. 2024-02-26T06:16:04.017 INFO:teuthology.orchestra.run.smithi006.stdout:mon is available 2024-02-26T06:16:04.017 INFO:teuthology.orchestra.run.smithi006.stdout:Assimilating anything we can from ceph.conf... 2024-02-26T06:16:04.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:04 smithi006 ceph-mon[31427]: from='client.? 172.21.15.6:0/4028291687' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T06:16:04.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:04 smithi006 ceph-mon[31427]: from='client.? 172.21.15.6:0/4028291687' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-26T06:16:05.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:05.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T06:16:05.356 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:05.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-02-26T06:16:05.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T06:16:05.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T06:16:05.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T06:16:05.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T06:16:05.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.621+0000 7f292d800700 1 Processor -- start 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.621+0000 7f292d800700 1 -- start start 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.622+0000 7f292d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f2928081950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:05.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.622+0000 7f292d800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2928081e90 con 0x7f2928081530 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.622+0000 7f2927e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f2928081950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.622+0000 7f2927e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f2928081950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51620/0 (socket says 172.21.15.6:51620) 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.622+0000 7f2927e00700 1 -- 172.21.15.6:0/1173353365 learned_addr learned my addr 172.21.15.6:0/1173353365 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.623+0000 7f2927e00700 1 -- 172.21.15.6:0/1173353365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928081fd0 con 0x7f2928081530 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.623+0000 7f2927e00700 1 --2- 172.21.15.6:0/1173353365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f2928081950 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f29180098a0 tx=0x7f2918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cabe9ce825702b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.623+0000 7f2927400700 1 -- 172.21.15.6:0/1173353365 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2918004160 con 0x7f2928081530 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f2927400700 1 -- 172.21.15.6:0/1173353365 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29180042c0 con 0x7f2928081530 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f2927400700 1 -- 172.21.15.6:0/1173353365 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29180045a0 con 0x7f2928081530 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f292d800700 1 -- 172.21.15.6:0/1173353365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 msgr2=0x7f2928081950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f292d800700 1 --2- 172.21.15.6:0/1173353365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f2928081950 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f29180098a0 tx=0x7f2918009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:05.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f292d800700 1 -- 172.21.15.6:0/1173353365 shutdown_connections 2024-02-26T06:16:05.364 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f292d800700 1 --2- 172.21.15.6:0/1173353365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f2928081950 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f292d800700 1 -- 172.21.15.6:0/1173353365 >> 172.21.15.6:0/1173353365 conn(0x7f292807cb90 msgr2=0x7f292807efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.624+0000 7f292d800700 1 -- 172.21.15.6:0/1173353365 shutdown_connections 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.625+0000 7f292d800700 1 -- 172.21.15.6:0/1173353365 wait complete. 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.625+0000 7f292d800700 1 Processor -- start 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.626+0000 7f292d800700 1 -- start start 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.626+0000 7f292d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f29281a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.626+0000 7f292d800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29281a1640 con 0x7f2928081530 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.626+0000 7f2927e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f29281a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.626+0000 7f2927e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f29281a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51630/0 (socket says 172.21.15.6:51630) 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.626+0000 7f2927e00700 1 -- 172.21.15.6:0/4028291687 learned_addr learned my addr 172.21.15.6:0/4028291687 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.627+0000 7f2927e00700 1 -- 172.21.15.6:0/4028291687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918009550 con 0x7f2928081530 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.627+0000 7f2927e00700 1 --2- 172.21.15.6:0/4028291687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f29281a1100 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2928082310 tx=0x7f2918004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:05.365 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.627+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2918011630 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.627+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918011790 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.627+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29281a1840 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.627+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2918011a30 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.628+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29281a1ce0 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.628+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2918022070 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.628+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f291801c450 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.629+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292819ad20 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.632+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2918011b90 con 0x7f2928081530 2024-02-26T06:16:05.366 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.709+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f292810ef70 con 0x7f2928081530 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.715+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f2918011de0 con 0x7f2928081530 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.716+0000 7f2925600700 1 -- 172.21.15.6:0/4028291687 <== mon.0 v2:172.21.15.6:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f291801c6b0 con 0x7f2928081530 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 msgr2=0x7f29281a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 --2- 172.21.15.6:0/4028291687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f29281a1100 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2928082310 tx=0x7f2918004750 comp rx=0 tx=0).stop 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 shutdown_connections 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 --2- 172.21.15.6:0/4028291687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2928081530 0x7f29281a1100 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 >> 172.21.15.6:0/4028291687 conn(0x7f292807cb90 msgr2=0x7f2928197e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 shutdown_connections 2024-02-26T06:16:05.367 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:04.717+0000 7f292d800700 1 -- 172.21.15.6:0/4028291687 wait complete. 2024-02-26T06:16:05.368 INFO:teuthology.orchestra.run.smithi006.stdout:Generating new minimal ceph.conf... 2024-02-26T06:16:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:06 smithi006 ceph-mon[31427]: from='client.? 172.21.15.6:0/3666644504' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.959+0000 7fc4fa400700 1 Processor -- start 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.959+0000 7fc4fa400700 1 -- start start 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.960+0000 7fc4fa400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f4109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.960+0000 7fc4fa400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f410a390 con 0x7fc4f41099a0 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.960+0000 7fc4f3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f4109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.960+0000 7fc4f3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f4109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51632/0 (socket says 172.21.15.6:51632) 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.960+0000 7fc4f3400700 1 -- 172.21.15.6:0/2059026729 learned_addr learned my addr 172.21.15.6:0/2059026729 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.961+0000 7fc4f3400700 1 -- 172.21.15.6:0/2059026729 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f410aba0 con 0x7fc4f41099a0 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.961+0000 7fc4f3400700 1 --2- 172.21.15.6:0/2059026729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f4109dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098a0 tx=0x7fc4e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9934790a23648d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.961+0000 7fc4f2000700 1 -- 172.21.15.6:0/2059026729 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4004160 con 0x7fc4f41099a0 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.961+0000 7fc4f2000700 1 -- 172.21.15.6:0/2059026729 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4e40042c0 con 0x7fc4f41099a0 2024-02-26T06:16:06.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.961+0000 7fc4f2000700 1 -- 172.21.15.6:0/2059026729 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4011ac0 con 0x7fc4f41099a0 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 -- 172.21.15.6:0/2059026729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 msgr2=0x7fc4f4109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 --2- 172.21.15.6:0/2059026729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f4109dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098a0 tx=0x7fc4e4009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 -- 172.21.15.6:0/2059026729 shutdown_connections 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 --2- 172.21.15.6:0/2059026729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f4109dc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 -- 172.21.15.6:0/2059026729 >> 172.21.15.6:0/2059026729 conn(0x7fc4f407cd60 msgr2=0x7fc4f41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 -- 172.21.15.6:0/2059026729 shutdown_connections 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.962+0000 7fc4fa400700 1 -- 172.21.15.6:0/2059026729 wait complete. 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.963+0000 7fc4fa400700 1 Processor -- start 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.964+0000 7fc4fa400700 1 -- start start 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.964+0000 7fc4fa400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.964+0000 7fc4fa400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f419dd20 con 0x7fc4f41099a0 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.964+0000 7fc4f3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.964+0000 7fc4f3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51648/0 (socket says 172.21.15.6:51648) 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.964+0000 7fc4f3400700 1 -- 172.21.15.6:0/3666644504 learned_addr learned my addr 172.21.15.6:0/3666644504 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:06.913 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4f3400700 1 -- 172.21.15.6:0/3666644504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4009550 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4f3400700 1 --2- 172.21.15.6:0/3666644504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f419d7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4011780 tx=0x7fc4e4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4003af0 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4e4003c50 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4e40197e0 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f419df20 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.965+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f419e3c0 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.966+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc4e4025460 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.966+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc4e400bb20 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.967+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f410ddd0 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:05.969+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc4e402a060 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.048+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc4f410dfe0 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.048+0000 7fc4ebe00700 1 -- 172.21.15.6:0/3666644504 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7fc4e4019d60 con 0x7fc4f41099a0 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.049+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 msgr2=0x7fc4f419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:06.914 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.050+0000 7fc4fa400700 1 --2- 172.21.15.6:0/3666644504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f419d7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4011780 tx=0x7fc4e4011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:06.915 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.050+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 shutdown_connections 2024-02-26T06:16:06.915 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.050+0000 7fc4fa400700 1 --2- 172.21.15.6:0/3666644504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f41099a0 0x7fc4f419d7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:06.915 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.050+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 >> 172.21.15.6:0/3666644504 conn(0x7fc4f407cd60 msgr2=0x7fc4f4105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:06.915 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.050+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 shutdown_connections 2024-02-26T06:16:06.915 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:06.050+0000 7fc4fa400700 1 -- 172.21.15.6:0/3666644504 wait complete. 2024-02-26T06:16:06.915 INFO:teuthology.orchestra.run.smithi006.stdout:Restarting the monitor... 2024-02-26T06:16:07.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:06 smithi006 systemd[1]: Stopping Ceph mon.a for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:16:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31423]: 2024-02-26T06:16:07.394+0000 7ff32a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:16:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31423]: 2024-02-26T06:16:07.394+0000 7ff32a000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-26T06:16:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:07 smithi006 podman[31668]: 2024-02-26 06:16:07.560756178 +0000 UTC m=+0.289481515 container died 81ca905c869a1e1bcabceee1b8d8500458bcf4bd1dcd05a4fd22f55e1f3a3554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-26T06:16:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 podman[31668]: 2024-02-26 06:16:08.146025442 +0000 UTC m=+0.874750775 container cleanup 81ca905c869a1e1bcabceee1b8d8500458bcf4bd1dcd05a4fd22f55e1f3a3554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-26T06:16:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 bash[31668]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a 2024-02-26T06:16:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 podman[31682]: 2024-02-26 06:16:08.346802635 +0000 UTC m=+0.769663831 container remove 81ca905c869a1e1bcabceee1b8d8500458bcf4bd1dcd05a4fd22f55e1f3a3554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:16:08.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-26T06:16:08.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 systemd[1]: Stopped Ceph mon.a for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:16:08.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service: Consumed 1.974s CPU time. 2024-02-26T06:16:09.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:08 smithi006 systemd[1]: Starting Ceph mon.a for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:16:09.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 podman[31771]: 2024-02-26 06:16:09.138078001 +0000 UTC m=+0.036200406 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:16:09.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 podman[31771]: 2024-02-26 06:16:09.28133746 +0000 UTC m=+0.179459881 container create daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-26T06:16:09.670 INFO:teuthology.orchestra.run.smithi006.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 podman[31771]: 2024-02-26 06:16:09.518549212 +0000 UTC m=+0.416671672 container init daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 podman[31771]: 2024-02-26 06:16:09.529909602 +0000 UTC m=+0.428032050 container start daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev), process ceph-mon, pid 2 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: pidfile_write: ignore empty --pid-file 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: load: jerasure load: lrc 2024-02-26T06:16:09.824 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: RocksDB version: 7.9.2 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Git sha 0 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Compile date 2024-02-16 06:18:30 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: DB SUMMARY 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: DB Session ID: MQ53RRM3PVL371YXJ88X 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: CURRENT file: CURRENT 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101668 ; 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.error_if_exists: 0 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.create_if_missing: 0 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.paranoid_checks: 1 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T06:16:09.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T06:16:09.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.env: 0x55ba0ef5bae0 2024-02-26T06:16:09.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T06:16:09.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.info_log: 0x55ba1154d900 2024-02-26T06:16:09.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T06:16:09.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.statistics: (nil) 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.use_fsync: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_log_file_size: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.allow_fallocate: 1 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.use_direct_reads: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T06:16:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.db_log_dir: 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.wal_dir: 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.write_buffer_manager: 0x55ba11521400 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T06:16:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.unordered_write: 0 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.row_cache: None 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.wal_filter: None 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T06:16:09.829 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.two_write_queues: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.wal_compression: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.atomic_flush: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.log_readahead_size: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T06:16:09.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_background_jobs: 2 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_background_compactions: -1 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_subcompactions: 1 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_open_files: -1 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T06:16:09.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_background_flushes: -1 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Compression algorithms supported: 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kZSTD supported: 0 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kXpressCompression supported: 0 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kZlibCompression supported: 1 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kSnappyCompression supported: 1 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kLZ4Compression supported: 1 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: kBZip2Compression supported: 0 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T06:16:09.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.merge_operator: 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_filter: None 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ba1154da00) 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks: 1 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_top_level_index_and_filter: 1 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_type: 0 2024-02-26T06:16:09.833 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_index_type: 0 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_shortening: 1 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: checksum: 4 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: no_block_cache: 0 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache: 0x55ba11551090 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_name: BinnedLRUCache 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_options: 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: capacity : 536870912 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_shard_bits : 4 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: strict_capacity_limit : 0 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: high_pri_pool_ratio: 0.000 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_compressed: (nil) 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: persistent_cache: (nil) 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size: 4096 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size_deviation: 10 2024-02-26T06:16:09.834 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_restart_interval: 16 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_block_restart_interval: 1 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: metadata_block_size: 4096 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: partition_filters: 0 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: use_delta_encoding: 1 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: filter_policy: bloomfilter 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: whole_key_filtering: 1 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: verify_compression: 0 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: read_amp_bytes_per_bit: 0 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: format_version: 5 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_index_compression: 1 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_align: 0 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: max_auto_readahead_size: 262144 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: prepopulate_block_cache: 0 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: initial_auto_readahead_size: 8192 2024-02-26T06:16:09.835 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T06:16:09.836 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression: NoCompression 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.num_levels: 7 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:16:09.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T06:16:09.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T06:16:09.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T06:16:09.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.table_properties_collectors: 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.inplace_update_support: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.bloom_locality: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.max_successive_merges: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.ttl: 2592000 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T06:16:09.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.enable_blob_files: false 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.min_blob_size: 0 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 075f12ea-cd45-442d-8882-2f2578c78f72 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928169587429, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-26T06:16:09.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928169589476, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 94641, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14263, "raw_average_key_size": 50, "raw_value_size": 86906, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708928169, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "075f12ea-cd45-442d-8882-2f2578c78f72", "db_session_id": "MQ53RRM3PVL371YXJ88X", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928169589623, "job": 1, "event": "recovery_finished"} 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ba11642000 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: DB pointer 0x55ba1162c000 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** DB Stats ** 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T06:16:09.843 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: L0 2/0 96.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: Sum 2/0 96.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 65.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-02-26T06:16:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative compaction: 0.00 GB write, 13.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval compaction: 0.00 GB write, 13.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache BinnedLRUCache@0x55ba11551090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T06:16:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: starting mon.a rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???) e1 preinit fsid 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).mds e1 new map 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).mds e1 print_map 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout: e1 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout: legacy client fscid: -1 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout: No filesystems configured 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T06:16:09.846 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: monmap e1: 1 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:16:09.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: fsmap 2024-02-26T06:16:09.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T06:16:09.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 ceph-mon[31790]: mgrmap e1: no daemons active 2024-02-26T06:16:09.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 bash[31771]: daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 2024-02-26T06:16:09.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:09 smithi006 systemd[1]: Started Ceph mon.a for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:16:11.133 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.342+0000 7f6bad400700 1 Processor -- start 2024-02-26T06:16:11.134 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.342+0000 7f6bad400700 1 -- start start 2024-02-26T06:16:11.134 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.343+0000 7f6bad400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba8107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:11.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.343+0000 7f6bad400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8108160 con 0x7f6ba8107770 2024-02-26T06:16:11.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.344+0000 7f6ba6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba8107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:11.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.344+0000 7f6ba6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba8107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54792/0 (socket says 172.21.15.6:54792) 2024-02-26T06:16:11.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.344+0000 7f6ba6800700 1 -- 172.21.15.6:0/384261915 learned_addr learned my addr 172.21.15.6:0/384261915 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.344+0000 7f6ba6800700 1 -- 172.21.15.6:0/384261915 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8108970 con 0x7f6ba8107770 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.345+0000 7f6ba6800700 1 --2- 172.21.15.6:0/384261915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba8107b90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098a0 tx=0x7f6b98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a68fff9dd91c0e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.345+0000 7f6ba5400700 1 -- 172.21.15.6:0/384261915 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b98004160 con 0x7f6ba8107770 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.345+0000 7f6ba5400700 1 -- 172.21.15.6:0/384261915 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b980042c0 con 0x7f6ba8107770 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.345+0000 7f6ba5400700 1 -- 172.21.15.6:0/384261915 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b98011ac0 con 0x7f6ba8107770 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 -- 172.21.15.6:0/384261915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 msgr2=0x7f6ba8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 --2- 172.21.15.6:0/384261915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba8107b90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098a0 tx=0x7f6b98009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 -- 172.21.15.6:0/384261915 shutdown_connections 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 --2- 172.21.15.6:0/384261915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba8107b90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 -- 172.21.15.6:0/384261915 >> 172.21.15.6:0/384261915 conn(0x7f6ba8102d10 msgr2=0x7f6ba8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 -- 172.21.15.6:0/384261915 shutdown_connections 2024-02-26T06:16:11.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.346+0000 7f6bad400700 1 -- 172.21.15.6:0/384261915 wait complete. 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.347+0000 7f6bad400700 1 Processor -- start 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.347+0000 7f6bad400700 1 -- start start 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.348+0000 7f6bad400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.348+0000 7f6bad400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819ba70 con 0x7f6ba8107770 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.348+0000 7f6ba6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.348+0000 7f6ba6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba819b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54808/0 (socket says 172.21.15.6:54808) 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.348+0000 7f6ba6800700 1 -- 172.21.15.6:0/3295962734 learned_addr learned my addr 172.21.15.6:0/3295962734 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6ba6800700 1 -- 172.21.15.6:0/3295962734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98009550 con 0x7f6ba8107770 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6ba6800700 1 --2- 172.21.15.6:0/3295962734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba819b530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b98011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b98003aa0 con 0x7f6ba8107770 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b98003c00 con 0x7f6ba8107770 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba819bc70 con 0x7f6ba8107770 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b980248a0 con 0x7f6ba8107770 2024-02-26T06:16:11.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.349+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba819c110 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.350+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6b9802d3f0 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.350+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6b980197b0 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.351+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba810bba0 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.356+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6b9801ac60 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.432+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6ba810bdb0 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.435+0000 7f6b9f400700 1 -- 172.21.15.6:0/3295962734 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6b980350c0 con 0x7f6ba8107770 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 msgr2=0x7f6ba819b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 --2- 172.21.15.6:0/3295962734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba819b530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b98011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 shutdown_connections 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 --2- 172.21.15.6:0/3295962734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba8107770 0x7f6ba819b530 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 >> 172.21.15.6:0/3295962734 conn(0x7f6ba8102d10 msgr2=0x7f6ba8103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 shutdown_connections 2024-02-26T06:16:11.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:10.437+0000 7f6bad400700 1 -- 172.21.15.6:0/3295962734 wait complete. 2024-02-26T06:16:11.142 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-26T06:16:11.164 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:16:11.164 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mgr... 2024-02-26T06:16:11.166 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-26T06:16:11.167 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-26T06:16:11.637 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x 2024-02-26T06:16:11.637 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x.service: Unit ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x.service not loaded. 2024-02-26T06:16:11.915 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5e92f702-d46e-11ee-95c3-87774f69a715.target.wants/ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-5e92f702-d46e-11ee-95c3-87774f69a715@.service. 2024-02-26T06:16:11.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:11 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3295962734' entity='client.admin' 2024-02-26T06:16:12.302 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:11 smithi006 systemd[1]: Starting Ceph mgr.x for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:16:12.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:12 smithi006 podman[32014]: 2024-02-26 06:16:12.301026123 +0000 UTC m=+0.112863764 container create 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T06:16:12.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:12 smithi006 podman[32014]: 2024-02-26 06:16:12.225559935 +0000 UTC m=+0.037397572 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T06:16:12.611 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:12 smithi006 podman[32014]: 2024-02-26 06:16:12.526973026 +0000 UTC m=+0.338810738 container init 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5) 2024-02-26T06:16:12.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:12 smithi006 podman[32014]: 2024-02-26 06:16:12.536334815 +0000 UTC m=+0.348172456 container start 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-26T06:16:12.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:12 smithi006 bash[32014]: 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 2024-02-26T06:16:12.640 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:16:12.640 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-02-26T06:16:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:16:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-02-26T06:16:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-02-26T06:16:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T06:16:12.673 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:16:12.674 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-02-26T06:16:12.688 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:16:12.689 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-02-26T06:16:12.689 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-02-26T06:16:12.689 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-26T06:16:12.689 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr to start... 2024-02-26T06:16:12.689 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr... 2024-02-26T06:16:12.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:12 smithi006 systemd[1]: Started Ceph mgr.x for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:16:13.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:13 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1281721622' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:16:13.672 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:13.354+0000 7f187be61200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:16:13.672 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:13.487+0000 7f187be61200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:16:14.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:14.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:16:14.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:16:14.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:16:14.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:16:14.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:14.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:16:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:16:14.091 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:16:14.092 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:16:02.582894+0000", 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.328+0000 7effc8c00700 1 Processor -- start 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.328+0000 7effc8c00700 1 -- start start 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.328+0000 7effc8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc4107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.328+0000 7effc8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4108170 con 0x7effc4107780 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.329+0000 7effc1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc4107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.329+0000 7effc1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc4107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54842/0 (socket says 172.21.15.6:54842) 2024-02-26T06:16:14.093 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.329+0000 7effc1e00700 1 -- 172.21.15.6:0/650561542 learned_addr learned my addr 172.21.15.6:0/650561542 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.330+0000 7effc1e00700 1 -- 172.21.15.6:0/650561542 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4108980 con 0x7effc4107780 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.330+0000 7effc1e00700 1 --2- 172.21.15.6:0/650561542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc4107ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7effb40098a0 tx=0x7effb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b42d79975a732ab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.330+0000 7effc0a00700 1 -- 172.21.15.6:0/650561542 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb4004160 con 0x7effc4107780 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.330+0000 7effc0a00700 1 -- 172.21.15.6:0/650561542 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effb40042c0 con 0x7effc4107780 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.330+0000 7effc0a00700 1 -- 172.21.15.6:0/650561542 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb4011ac0 con 0x7effc4107780 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 -- 172.21.15.6:0/650561542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 msgr2=0x7effc4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 --2- 172.21.15.6:0/650561542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc4107ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7effb40098a0 tx=0x7effb4009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 -- 172.21.15.6:0/650561542 shutdown_connections 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 --2- 172.21.15.6:0/650561542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc4107ba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 -- 172.21.15.6:0/650561542 >> 172.21.15.6:0/650561542 conn(0x7effc4102d80 msgr2=0x7effc41051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 -- 172.21.15.6:0/650561542 shutdown_connections 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.331+0000 7effc8c00700 1 -- 172.21.15.6:0/650561542 wait complete. 2024-02-26T06:16:14.094 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.332+0000 7effc8c00700 1 Processor -- start 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.332+0000 7effc8c00700 1 -- start start 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.333+0000 7effc8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc407ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.333+0000 7effc8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc407cf80 con 0x7effc4107780 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.333+0000 7effc1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc407ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.333+0000 7effc1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc407ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54844/0 (socket says 172.21.15.6:54844) 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.333+0000 7effc1e00700 1 -- 172.21.15.6:0/1281721622 learned_addr learned my addr 172.21.15.6:0/1281721622 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.333+0000 7effc1e00700 1 -- 172.21.15.6:0/1281721622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4009550 con 0x7effc4107780 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.334+0000 7effc1e00700 1 --2- 172.21.15.6:0/1281721622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc407ca40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effb4011780 tx=0x7effb4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.334+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb4003af0 con 0x7effc4107780 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.334+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effb4003c50 con 0x7effc4107780 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.334+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc407b170 con 0x7effc4107780 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.334+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40197e0 con 0x7effc4107780 2024-02-26T06:16:14.095 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.334+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc407b610 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.335+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7effb40199d0 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.335+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7effb400b720 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.335+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc40632a0 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.338+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7effb402a060 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.418+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7effc410bdc0 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.419+0000 7effbaa00700 1 -- 172.21.15.6:0/1281721622 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7effb401aba0 con 0x7effc4107780 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.420+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 msgr2=0x7effc407ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.420+0000 7effc8c00700 1 --2- 172.21.15.6:0/1281721622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc407ca40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effb4011780 tx=0x7effb4011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.420+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 shutdown_connections 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.420+0000 7effc8c00700 1 --2- 172.21.15.6:0/1281721622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc4107780 0x7effc407ca40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.420+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 >> 172.21.15.6:0/1281721622 conn(0x7effc4102d80 msgr2=0x7effc4103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.421+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 shutdown_connections 2024-02-26T06:16:14.096 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:13.421+0000 7effc8c00700 1 -- 172.21.15.6:0/1281721622 wait complete. 2024-02-26T06:16:14.097 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (1/15)... 2024-02-26T06:16:14.626 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:14.320+0000 7f187be61200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:16:14.626 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:14.446+0000 7f187be61200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:16:14.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:14.626+0000 7f187be61200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:16:14.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:14.754+0000 7f187be61200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:16:14.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:14.866+0000 7f187be61200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:16:15.468 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:15.209+0000 7f187be61200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:16:15.468 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:15.352+0000 7f187be61200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:16:15.817 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:15.468+0000 7f187be61200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:16:15.817 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:15.594+0000 7f187be61200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:16:15.818 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:15.817+0000 7f187be61200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:16:17.108 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:16 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/748416931' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:16:17.402 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:17.225+0000 7f187be61200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:16:17.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:17.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:16:17.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:16:17.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:16:17.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:16:17.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:16:17.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:16:17.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:16:17.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.441 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.442 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:16:02.582894+0000", 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.665+0000 7f7cd0800700 1 Processor -- start 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.666+0000 7f7cd0800700 1 -- start start 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.666+0000 7f7cd0800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc8106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:17.443 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.666+0000 7f7cd0800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc8107070 con 0x7f7cc8106680 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.667+0000 7f7ccde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc8106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.667+0000 7f7ccde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc8106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54860/0 (socket says 172.21.15.6:54860) 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.667+0000 7f7ccde00700 1 -- 172.21.15.6:0/591861768 learned_addr learned my addr 172.21.15.6:0/591861768 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.667+0000 7f7ccde00700 1 -- 172.21.15.6:0/591861768 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc8107880 con 0x7f7cc8106680 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.667+0000 7f7ccde00700 1 --2- 172.21.15.6:0/591861768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc8106aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00bb40 tx=0x7f7cbc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e963d8458ea4f7b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.668+0000 7f7ccca00700 1 -- 172.21.15.6:0/591861768 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cbc00dbe0 con 0x7f7cc8106680 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.668+0000 7f7ccca00700 1 -- 172.21.15.6:0/591861768 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cbc0044f0 con 0x7f7cc8106680 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.668+0000 7f7ccca00700 1 -- 172.21.15.6:0/591861768 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cbc010470 con 0x7f7cc8106680 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 -- 172.21.15.6:0/591861768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 msgr2=0x7f7cc8106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 --2- 172.21.15.6:0/591861768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc8106aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00bb40 tx=0x7f7cbc00be50 comp rx=0 tx=0).stop 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 -- 172.21.15.6:0/591861768 shutdown_connections 2024-02-26T06:16:17.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 --2- 172.21.15.6:0/591861768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc8106aa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 -- 172.21.15.6:0/591861768 >> 172.21.15.6:0/591861768 conn(0x7f7cc8101c40 msgr2=0x7f7cc8104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 -- 172.21.15.6:0/591861768 shutdown_connections 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.669+0000 7f7cd0800700 1 -- 172.21.15.6:0/591861768 wait complete. 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.670+0000 7f7cd0800700 1 Processor -- start 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.670+0000 7f7cd0800700 1 -- start start 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.671+0000 7f7cd0800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.671+0000 7f7cd0800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc81998f0 con 0x7f7cc8106680 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.671+0000 7f7ccde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.671+0000 7f7ccde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc81993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54874/0 (socket says 172.21.15.6:54874) 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.671+0000 7f7ccde00700 1 -- 172.21.15.6:0/748416931 learned_addr learned my addr 172.21.15.6:0/748416931 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.672+0000 7f7ccde00700 1 -- 172.21.15.6:0/748416931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc00b7f0 con 0x7f7cc8106680 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.672+0000 7f7ccde00700 1 --2- 172.21.15.6:0/748416931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc81993b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b340 tx=0x7f7cbc010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.672+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cbc01e070 con 0x7f7cc8106680 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.672+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cbc004250 con 0x7f7cc8106680 2024-02-26T06:16:17.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.672+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cbc022a80 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.672+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc8199af0 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.673+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc8199f90 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.673+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7cbc022be0 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.673+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cbc019690 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.674+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc80632a0 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.676+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7cbc027070 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.755+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7cc8192a80 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.756+0000 7f7cc2a00700 1 -- 172.21.15.6:0/748416931 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7cbc010df0 con 0x7f7cc8106680 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 msgr2=0x7f7cc81993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 --2- 172.21.15.6:0/748416931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc81993b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b340 tx=0x7f7cbc010520 comp rx=0 tx=0).stop 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 shutdown_connections 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 --2- 172.21.15.6:0/748416931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc8106680 0x7f7cc81993b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 >> 172.21.15.6:0/748416931 conn(0x7f7cc8101c40 msgr2=0x7f7cc818fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 shutdown_connections 2024-02-26T06:16:17.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:16.757+0000 7f7cd0800700 1 -- 172.21.15.6:0/748416931 wait complete. 2024-02-26T06:16:17.447 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (2/15)... 2024-02-26T06:16:17.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:17.402+0000 7f187be61200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:16:18.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:17.829+0000 7f187be61200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:16:18.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:17.949+0000 7f187be61200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:16:18.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:18.212+0000 7f187be61200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:16:18.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:18.460+0000 7f187be61200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:16:19.163 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:18.840+0000 7f187be61200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:16:19.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:19.164+0000 7f187be61200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:16:19.733 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:19.475+0000 7f187be61200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:16:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:20 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3194644372' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:16:20.806 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:20.576+0000 7f187be61200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:16:20.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:20.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:16:20.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:16:20.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:20.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:16:20.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:16:20.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:16:02.582894+0000", 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.965+0000 7ff589000700 1 Processor -- start 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.965+0000 7ff589000700 1 -- start start 2024-02-26T06:16:20.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.966+0000 7ff589000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff584107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.966+0000 7ff589000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584108170 con 0x7ff584107780 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.966+0000 7ff581e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff584107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.966+0000 7ff581e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff584107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50398/0 (socket says 172.21.15.6:50398) 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.966+0000 7ff581e00700 1 -- 172.21.15.6:0/3163129644 learned_addr learned my addr 172.21.15.6:0/3163129644 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.967+0000 7ff581e00700 1 -- 172.21.15.6:0/3163129644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584108980 con 0x7ff584107780 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.967+0000 7ff581e00700 1 --2- 172.21.15.6:0/3163129644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff584107ba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098a0 tx=0x7ff574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3a8ce1dd07c462 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.967+0000 7ff580a00700 1 -- 172.21.15.6:0/3163129644 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff574004160 con 0x7ff584107780 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.967+0000 7ff580a00700 1 -- 172.21.15.6:0/3163129644 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5740042c0 con 0x7ff584107780 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.967+0000 7ff580a00700 1 -- 172.21.15.6:0/3163129644 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff574011ac0 con 0x7ff584107780 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 -- 172.21.15.6:0/3163129644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 msgr2=0x7ff584107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 --2- 172.21.15.6:0/3163129644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff584107ba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098a0 tx=0x7ff574009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 -- 172.21.15.6:0/3163129644 shutdown_connections 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 --2- 172.21.15.6:0/3163129644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff584107ba0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 -- 172.21.15.6:0/3163129644 >> 172.21.15.6:0/3163129644 conn(0x7ff584102d80 msgr2=0x7ff5841051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 -- 172.21.15.6:0/3163129644 shutdown_connections 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.968+0000 7ff589000700 1 -- 172.21.15.6:0/3163129644 wait complete. 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.969+0000 7ff589000700 1 Processor -- start 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.970+0000 7ff589000700 1 -- start start 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.970+0000 7ff589000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff58407ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.970+0000 7ff589000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58407cf80 con 0x7ff584107780 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.970+0000 7ff581e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff58407ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.970+0000 7ff581e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff58407ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50406/0 (socket says 172.21.15.6:50406) 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.970+0000 7ff581e00700 1 -- 172.21.15.6:0/3194644372 learned_addr learned my addr 172.21.15.6:0/3194644372 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.971+0000 7ff581e00700 1 -- 172.21.15.6:0/3194644372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574009550 con 0x7ff584107780 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.971+0000 7ff581e00700 1 --2- 172.21.15.6:0/3194644372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff58407ca40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff574011780 tx=0x7ff574011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:20.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.971+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff574003af0 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.971+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff574003c50 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.971+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58407b170 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.971+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5740197e0 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.972+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58407b610 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.972+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564005360 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.973+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff5740199d0 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.973+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff57400b720 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:19.975+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff57402a060 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.055+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff5640058e0 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.055+0000 7ff57aa00700 1 -- 172.21.15.6:0/3194644372 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7ff574019c80 con 0x7ff584107780 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 msgr2=0x7ff58407ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 --2- 172.21.15.6:0/3194644372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff58407ca40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff574011780 tx=0x7ff574011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:20.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 shutdown_connections 2024-02-26T06:16:20.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 --2- 172.21.15.6:0/3194644372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff584107780 0x7ff58407ca40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:20.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 >> 172.21.15.6:0/3194644372 conn(0x7ff584102d80 msgr2=0x7ff584103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:20.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 shutdown_connections 2024-02-26T06:16:20.839 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:20.058+0000 7ff589000700 1 -- 172.21.15.6:0/3194644372 wait complete. 2024-02-26T06:16:20.839 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (3/15)... 2024-02-26T06:16:21.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:21.294+0000 7f187be61200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:16:21.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:21.425+0000 7f187be61200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:16:22.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:21.711+0000 7f187be61200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: Activating manager daemon x 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: mgrmap e2: x(active, starting, since 0.0096924s) 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: Manager daemon x is now available 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:16:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' 2024-02-26T06:16:22.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:16:22.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:21 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' 2024-02-26T06:16:23.127 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:22 smithi006 ceph-mon[31790]: from='mgr.14100 172.21.15.6:0/2868645488' entity='mgr.x' 2024-02-26T06:16:23.127 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:22 smithi006 ceph-mon[31790]: mgrmap e3: x(active, since 1.02066s) 2024-02-26T06:16:24.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:24 smithi006 ceph-mon[31790]: mgrmap e4: x(active, since 2s) 2024-02-26T06:16:24.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:24 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/691666561' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T06:16:24.917 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:24.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:16:24.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:16:24.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T06:16:24.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T06:16:24.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T06:16:24.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T06:16:24.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T06:16:24.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T06:16:24.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-02-26T06:16:24.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:24.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T06:16:24.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T06:16:24.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:24.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-26T06:16:24.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T06:16:24.922 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:24.922 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T06:16:24.922 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T06:16:24.922 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.923 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T06:16:24.923 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:24.923 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T06:16:24.923 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T06:16:24.924 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T06:16:24.924 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T06:16:24.924 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T06:16:24.924 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T06:16:24.924 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.925 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T06:16:24.925 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T06:16:24.925 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T06:16:24.925 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.927 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T06:16:24.928 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:24.928 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T06:16:24.928 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T06:16:24.928 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.928 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T06:16:24.928 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T06:16:24.929 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T06:16:24.929 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T06:16:24.929 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T06:16:24.929 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T06:16:24.929 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T06:16:02.582894+0000", 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T06:16:24.931 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.450+0000 7f4784400700 1 Processor -- start 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.450+0000 7f4784400700 1 -- start start 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.450+0000 7f4784400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c081bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.450+0000 7f4784400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c082180 con 0x7f477c081790 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.451+0000 7f4781a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c081bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.451+0000 7f4781a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c081bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50460/0 (socket says 172.21.15.6:50460) 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.451+0000 7f4781a00700 1 -- 172.21.15.6:0/2613334547 learned_addr learned my addr 172.21.15.6:0/2613334547 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.452+0000 7f4781a00700 1 -- 172.21.15.6:0/2613334547 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c10ef60 con 0x7f477c081790 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.452+0000 7f4781a00700 1 --2- 172.21.15.6:0/2613334547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c081bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f47700098a0 tx=0x7f4770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e14ccbbb6ede9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.452+0000 7f477be00700 1 -- 172.21.15.6:0/2613334547 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4770004160 con 0x7f477c081790 2024-02-26T06:16:24.933 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.452+0000 7f477be00700 1 -- 172.21.15.6:0/2613334547 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47700042c0 con 0x7f477c081790 2024-02-26T06:16:24.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.452+0000 7f477be00700 1 -- 172.21.15.6:0/2613334547 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4770011ac0 con 0x7f477c081790 2024-02-26T06:16:24.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 -- 172.21.15.6:0/2613334547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 msgr2=0x7f477c081bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:24.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 --2- 172.21.15.6:0/2613334547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c081bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f47700098a0 tx=0x7f4770009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:24.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 -- 172.21.15.6:0/2613334547 shutdown_connections 2024-02-26T06:16:24.935 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 --2- 172.21.15.6:0/2613334547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c081bb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 -- 172.21.15.6:0/2613334547 >> 172.21.15.6:0/2613334547 conn(0x7f477c07cd50 msgr2=0x7f477c07f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 -- 172.21.15.6:0/2613334547 shutdown_connections 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.453+0000 7f4784400700 1 -- 172.21.15.6:0/2613334547 wait complete. 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.454+0000 7f4784400700 1 Processor -- start 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.454+0000 7f4784400700 1 -- start start 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.455+0000 7f4784400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.455+0000 7f4784400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1a2080 con 0x7f477c081790 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.455+0000 7f4781a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.455+0000 7f4781a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c1a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50476/0 (socket says 172.21.15.6:50476) 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.455+0000 7f4781a00700 1 -- 172.21.15.6:0/691666561 learned_addr learned my addr 172.21.15.6:0/691666561 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:24.936 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f4781a00700 1 -- 172.21.15.6:0/691666561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4770009550 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f4781a00700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c1a1b40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4770000c00 tx=0x7f4770011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4770003aa0 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4770003c00 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47700248a0 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c1a2280 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.456+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c1a2720 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.458+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 3) v1 ==== 45815+0+0 (secure 0 0 0) 0x7f477001ac60 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.458+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c112190 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.458+0000 7f477a000700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f4764038dc0 0x7f476403b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.458+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f477004ca90 con 0x7f477c081790 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.458+0000 7f4781000700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f4764038dc0 0x7f476403b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.459+0000 7f4781000700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f4764038dc0 0x7f476403b280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4768006fd0 tx=0x7f4768006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:24.939 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.465+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4770018530 con 0x7f477c081790 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.778+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f477002d770 con 0x7f477c081790 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.783+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f477c1a5390 con 0x7f477c081790 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.784+0000 7f477a000700 1 -- 172.21.15.6:0/691666561 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f4770018920 con 0x7f477c081790 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f4764038dc0 msgr2=0x7f476403b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f4764038dc0 0x7f476403b280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4768006fd0 tx=0x7f4768006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 msgr2=0x7f477c1a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:24.940 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c1a1b40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4770000c00 tx=0x7f4770011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 shutdown_connections 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f4764038dc0 0x7f476403b280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 --2- 172.21.15.6:0/691666561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f477c081790 0x7f477c1a1b40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 >> 172.21.15.6:0/691666561 conn(0x7f477c07cd50 msgr2=0x7f477c07d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 shutdown_connections 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:23.787+0000 7f4784400700 1 -- 172.21.15.6:0/691666561 wait complete. 2024-02-26T06:16:24.941 INFO:teuthology.orchestra.run.smithi006.stdout:mgr is available 2024-02-26T06:16:26.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:25 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1855984281' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T06:16:26.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:26.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T06:16:26.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:26.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T06:16:26.527 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.594+0000 7f512bc00700 1 Processor -- start 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.595+0000 7f512bc00700 1 -- start start 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.595+0000 7f512bc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512407b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.595+0000 7f512bc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512407b7e0 con 0x7f512407cdb0 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.596+0000 7f5129200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512407b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.596+0000 7f5129200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512407b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50480/0 (socket says 172.21.15.6:50480) 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.596+0000 7f5129200700 1 -- 172.21.15.6:0/2956940626 learned_addr learned my addr 172.21.15.6:0/2956940626 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.596+0000 7f5129200700 1 -- 172.21.15.6:0/2956940626 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512407b920 con 0x7f512407cdb0 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.597+0000 7f5129200700 1 --2- 172.21.15.6:0/2956940626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512407b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51180098a0 tx=0x7f5118009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa6168ea39021a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.597+0000 7f5123400700 1 -- 172.21.15.6:0/2956940626 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5118004160 con 0x7f512407cdb0 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.597+0000 7f5123400700 1 -- 172.21.15.6:0/2956940626 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51180042c0 con 0x7f512407cdb0 2024-02-26T06:16:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.597+0000 7f5123400700 1 -- 172.21.15.6:0/2956940626 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5118011ac0 con 0x7f512407cdb0 2024-02-26T06:16:26.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.598+0000 7f512bc00700 1 -- 172.21.15.6:0/2956940626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 msgr2=0x7f512407b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:26.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.598+0000 7f512bc00700 1 --2- 172.21.15.6:0/2956940626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512407b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51180098a0 tx=0x7f5118009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:26.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.598+0000 7f512bc00700 1 -- 172.21.15.6:0/2956940626 shutdown_connections 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.598+0000 7f512bc00700 1 --2- 172.21.15.6:0/2956940626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512407b210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.598+0000 7f512bc00700 1 -- 172.21.15.6:0/2956940626 >> 172.21.15.6:0/2956940626 conn(0x7f5124102d80 msgr2=0x7f51241051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.599+0000 7f512bc00700 1 -- 172.21.15.6:0/2956940626 shutdown_connections 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.599+0000 7f512bc00700 1 -- 172.21.15.6:0/2956940626 wait complete. 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.600+0000 7f512bc00700 1 Processor -- start 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.600+0000 7f512bc00700 1 -- start start 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.601+0000 7f512bc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.601+0000 7f512bc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512419dd00 con 0x7f512407cdb0 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.601+0000 7f5129200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.601+0000 7f5129200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50492/0 (socket says 172.21.15.6:50492) 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.601+0000 7f5129200700 1 -- 172.21.15.6:0/1855984281 learned_addr learned my addr 172.21.15.6:0/1855984281 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.601+0000 7f5129200700 1 -- 172.21.15.6:0/1855984281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009550 con 0x7f512407cdb0 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.602+0000 7f5129200700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512419d7c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5118011780 tx=0x7f5118011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.602+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5118003af0 con 0x7f512407cdb0 2024-02-26T06:16:26.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.602+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5118003c50 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.602+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512419df00 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.602+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51180197e0 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.602+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512419e3a0 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.604+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f511801aba0 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.604+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512410dbc0 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.604+0000 7f5121600700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f510c038e90 0x7f510c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.604+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f511804d270 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.604+0000 7f5123e00700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f510c038e90 0x7f510c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.605+0000 7f5123e00700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f510c038e90 0x7f510c03b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5110006fd0 tx=0x7f5110006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.611+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f511801a430 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.833+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f51241a0e10 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.834+0000 7f5121600700 1 -- 172.21.15.6:0/1855984281 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f511802dbe0 con 0x7f512407cdb0 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f510c038e90 msgr2=0x7f510c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f510c038e90 0x7f510c03b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5110006fd0 tx=0x7f5110006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:26.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 msgr2=0x7f512419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512419d7c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5118011780 tx=0x7f5118011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 shutdown_connections 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f510c038e90 0x7f510c03b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 --2- 172.21.15.6:0/1855984281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512407cdb0 0x7f512419d7c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.837+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 >> 172.21.15.6:0/1855984281 conn(0x7f5124102d80 msgr2=0x7f51241038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.838+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 shutdown_connections 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:25.838+0000 7f512bc00700 1 -- 172.21.15.6:0/1855984281 wait complete. 2024-02-26T06:16:26.535 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling cephadm module... 2024-02-26T06:16:27.881 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:27 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3972338784' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-26T06:16:27.882 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: ignoring --setuser ceph since I am not root 2024-02-26T06:16:27.882 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: ignoring --setgroup ceph since I am not root 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.140+0000 7fc952a00700 1 Processor -- start 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.140+0000 7fc952a00700 1 -- start start 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.141+0000 7fc952a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.141+0000 7fc952a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c07c8f0 con 0x7fc94c07dec0 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.141+0000 7fc94be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.141+0000 7fc94be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45388/0 (socket says 172.21.15.6:45388) 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.141+0000 7fc94be00700 1 -- 172.21.15.6:0/1394204943 learned_addr learned my addr 172.21.15.6:0/1394204943 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.142+0000 7fc94be00700 1 -- 172.21.15.6:0/1394204943 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c07ca30 con 0x7fc94c07dec0 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.142+0000 7fc94be00700 1 --2- 172.21.15.6:0/1394204943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c07c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098a0 tx=0x7fc93c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e443890a8b5f48 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.142+0000 7fc94b400700 1 -- 172.21.15.6:0/1394204943 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc93c004160 con 0x7fc94c07dec0 2024-02-26T06:16:28.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.142+0000 7fc94b400700 1 -- 172.21.15.6:0/1394204943 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc93c0042c0 con 0x7fc94c07dec0 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.142+0000 7fc94b400700 1 -- 172.21.15.6:0/1394204943 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc93c011ac0 con 0x7fc94c07dec0 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 -- 172.21.15.6:0/1394204943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 msgr2=0x7fc94c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 --2- 172.21.15.6:0/1394204943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c07c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098a0 tx=0x7fc93c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 -- 172.21.15.6:0/1394204943 shutdown_connections 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 --2- 172.21.15.6:0/1394204943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c07c320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 -- 172.21.15.6:0/1394204943 >> 172.21.15.6:0/1394204943 conn(0x7fc94c102d60 msgr2=0x7fc94c105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 -- 172.21.15.6:0/1394204943 shutdown_connections 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.143+0000 7fc952a00700 1 -- 172.21.15.6:0/1394204943 wait complete. 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.144+0000 7fc952a00700 1 Processor -- start 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.145+0000 7fc952a00700 1 -- start start 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.145+0000 7fc952a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.145+0000 7fc952a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c19dc80 con 0x7fc94c07dec0 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.145+0000 7fc94be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:28.207 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.145+0000 7fc94be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45396/0 (socket says 172.21.15.6:45396) 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.145+0000 7fc94be00700 1 -- 172.21.15.6:0/3972338784 learned_addr learned my addr 172.21.15.6:0/3972338784 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc94be00700 1 -- 172.21.15.6:0/3972338784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c009550 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc94be00700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c19d740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc93c000c00 tx=0x7fc93c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc93c003aa0 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc93c003c00 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c19de80 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc93c0248a0 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.146+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c19e320 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.148+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7fc93c01a430 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.148+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94c10dbc0 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.148+0000 7fc949600700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fc930038e90 0x7fc93003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.148+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc93c04cce0 con 0x7fc94c07dec0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.148+0000 7fc943400700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fc930038e90 0x7fc93003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:28.208 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.149+0000 7fc943400700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fc930038e90 0x7fc93003b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc93400ad30 tx=0x7fc9340093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.153+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc93c018710 con 0x7fc94c07dec0 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.438+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc94c1a10f0 con 0x7fc94c07dec0 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.492+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc93c018350 con 0x7fc94c07dec0 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc949600700 1 -- 172.21.15.6:0/3972338784 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fc93c021d60 con 0x7fc94c07dec0 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fc930038e90 msgr2=0x7fc93003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fc930038e90 0x7fc93003b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc93400ad30 tx=0x7fc9340093f0 comp rx=0 tx=0).stop 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 msgr2=0x7fc94c19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c19d740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc93c000c00 tx=0x7fc93c011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 shutdown_connections 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fc930038e90 0x7fc93003b350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 --2- 172.21.15.6:0/3972338784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc94c07dec0 0x7fc94c19d740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.496+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 >> 172.21.15.6:0/3972338784 conn(0x7fc94c102d60 msgr2=0x7fc94c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.498+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 shutdown_connections 2024-02-26T06:16:28.209 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:27.498+0000 7fc952a00700 1 -- 172.21.15.6:0/3972338784 wait complete. 2024-02-26T06:16:28.500 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:28 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:28.288+0000 7fd157993200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:16:28.500 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:28 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:28.423+0000 7fd157993200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:16:28.759 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:28 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3972338784' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-26T06:16:28.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:28 smithi006 ceph-mon[31790]: mgrmap e5: x(active, since 5s) 2024-02-26T06:16:29.394 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:29.267+0000 7fd157993200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:16:29.394 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:29.394+0000 7fd157993200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:16:29.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:29 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3357287793' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T06:16:29.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:29.577+0000 7fd157993200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:16:29.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:16:29.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-26T06:16:29.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T06:16:29.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T06:16:29.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T06:16:29.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:16:29.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.817+0000 7fe0a4800700 1 Processor -- start 2024-02-26T06:16:29.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.817+0000 7fe0a4800700 1 -- start start 2024-02-26T06:16:29.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.817+0000 7fe0a4800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c07b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:29.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.817+0000 7fe0a4800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c07b7e0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.818+0000 7fe0a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c07b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:29.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.818+0000 7fe0a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c07b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45430/0 (socket says 172.21.15.6:45430) 2024-02-26T06:16:29.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.818+0000 7fe0a1e00700 1 -- 172.21.15.6:0/1124167336 learned_addr learned my addr 172.21.15.6:0/1124167336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:29.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.818+0000 7fe0a1e00700 1 -- 172.21.15.6:0/1124167336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c07b920 con 0x7fe09c07cdb0 2024-02-26T06:16:29.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.819+0000 7fe0a1e00700 1 --2- 172.21.15.6:0/1124167336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c07b210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe0900098a0 tx=0x7fe090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe6546a75ca4c07 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:29.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.819+0000 7fe0a0a00700 1 -- 172.21.15.6:0/1124167336 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe090004160 con 0x7fe09c07cdb0 2024-02-26T06:16:29.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.819+0000 7fe0a0a00700 1 -- 172.21.15.6:0/1124167336 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0900042c0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.819+0000 7fe0a0a00700 1 -- 172.21.15.6:0/1124167336 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe090011ac0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.819+0000 7fe0a4800700 1 -- 172.21.15.6:0/1124167336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 msgr2=0x7fe09c07b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:29.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.819+0000 7fe0a4800700 1 --2- 172.21.15.6:0/1124167336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c07b210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe0900098a0 tx=0x7fe090009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:29.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.820+0000 7fe0a4800700 1 -- 172.21.15.6:0/1124167336 shutdown_connections 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.820+0000 7fe0a4800700 1 --2- 172.21.15.6:0/1124167336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c07b210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.820+0000 7fe0a4800700 1 -- 172.21.15.6:0/1124167336 >> 172.21.15.6:0/1124167336 conn(0x7fe09c102d80 msgr2=0x7fe09c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.820+0000 7fe0a4800700 1 -- 172.21.15.6:0/1124167336 shutdown_connections 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.820+0000 7fe0a4800700 1 -- 172.21.15.6:0/1124167336 wait complete. 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.821+0000 7fe0a4800700 1 Processor -- start 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.821+0000 7fe0a4800700 1 -- start start 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.821+0000 7fe0a4800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.821+0000 7fe0a4800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a20f0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.822+0000 7fe0a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.822+0000 7fe0a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c1a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45446/0 (socket says 172.21.15.6:45446) 2024-02-26T06:16:29.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.822+0000 7fe0a1e00700 1 -- 172.21.15.6:0/3357287793 learned_addr learned my addr 172.21.15.6:0/3357287793 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:29.881 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.822+0000 7fe0a1e00700 1 -- 172.21.15.6:0/3357287793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090009550 con 0x7fe09c07cdb0 2024-02-26T06:16:29.881 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.822+0000 7fe0a1e00700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c1a1bb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe090011780 tx=0x7fe090011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:29.881 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.823+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe090003af0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.881 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.823+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe090003c50 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.823+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0900197e0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.823+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c1a22f0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.823+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1a2790 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.824+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fe09001aba0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.824+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c1097b0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.825+0000 7fe096a00700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 0x7fe08403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.825+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe09004d2d0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.825+0000 7fe0a1400700 1 -- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 msgr2=0x7fe08403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.825+0000 7fe0a1400700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 0x7fe08403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:28.834+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe09001a430 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.026+0000 7fe0a1400700 1 -- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 msgr2=0x7fe08403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.026+0000 7fe0a1400700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 0x7fe08403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.113+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe09c19b280 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.114+0000 7fe096a00700 1 -- 172.21.15.6:0/3357287793 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe09002dbe0 con 0x7fe09c07cdb0 2024-02-26T06:16:29.882 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 msgr2=0x7fe08403b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 0x7fe08403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 msgr2=0x7fe09c1a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c1a1bb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe090011780 tx=0x7fe090011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 shutdown_connections 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7fe084038ee0 0x7fe08403b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 --2- 172.21.15.6:0/3357287793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe09c07cdb0 0x7fe09c1a1bb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 >> 172.21.15.6:0/3357287793 conn(0x7fe09c102d80 msgr2=0x7fe09c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:29.883 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 shutdown_connections 2024-02-26T06:16:29.884 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:29.117+0000 7fe0a4800700 1 -- 172.21.15.6:0/3357287793 wait complete. 2024-02-26T06:16:29.884 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-02-26T06:16:29.884 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 5... 2024-02-26T06:16:30.034 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:29.705+0000 7fd157993200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:16:30.034 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:29.817+0000 7fd157993200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:16:30.313 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:30.168+0000 7fd157993200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:16:30.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:30.313+0000 7fd157993200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:16:30.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:30.435+0000 7fd157993200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:16:30.612 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:30.560+0000 7fd157993200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:16:30.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:30.784+0000 7fd157993200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:16:32.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:32 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:32.097+0000 7fd157993200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:16:32.422 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:32 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:32.275+0000 7fd157993200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:16:33.087 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:32 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:32.700+0000 7fd157993200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:16:33.087 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:32 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:32.822+0000 7fd157993200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:16:33.087 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:33 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:33.087+0000 7fd157993200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:16:33.416 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:33 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:33.336+0000 7fd157993200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:16:34.043 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:33 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:33.718+0000 7fd157993200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:16:34.044 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:34 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:34.043+0000 7fd157993200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:16:34.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:34 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:34.354+0000 7fd157993200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:16:35.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:35 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:35.413+0000 7fd157993200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:16:36.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:36.130+0000 7fd157993200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:16:36.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:36.263+0000 7fd157993200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:16:36.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:16:36.551+0000 7fd157993200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: Active manager daemon x restarted 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: Activating manager daemon x 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: osdmap e2: 0 total, 0 up, 0 in 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: mgrmap e6: x(active, starting, since 0.00461761s) 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:16:36.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:36 smithi006 ceph-mon[31790]: Manager daemon x is now available 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: Found migration_current of "None". Setting to last migration. 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:16:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:37.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:37.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: [26/Feb/2024:06:16:37] ENGINE Bus STARTING 2024-02-26T06:16:37.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T06:16:37.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: mgrmap e7: x(active, since 1.0131s) 2024-02-26T06:16:37.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:37 smithi006 ceph-mon[31790]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.403+0000 7f5307400700 1 Processor -- start 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.403+0000 7f5307400700 1 -- start start 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.404+0000 7f5307400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.404+0000 7f5307400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530007c870 con 0x7f530007de40 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.404+0000 7f5304a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.404+0000 7f5304a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45448/0 (socket says 172.21.15.6:45448) 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.404+0000 7f5304a00700 1 -- 172.21.15.6:0/1243662155 learned_addr learned my addr 172.21.15.6:0/1243662155 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.404+0000 7f5304a00700 1 -- 172.21.15.6:0/1243662155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530007c9b0 con 0x7f530007de40 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.405+0000 7f5304a00700 1 --2- 172.21.15.6:0/1243662155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530007c2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52f40098a0 tx=0x7f52f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5d2f52602aad03 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.405+0000 7f52ff400700 1 -- 172.21.15.6:0/1243662155 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f4004160 con 0x7f530007de40 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.405+0000 7f52ff400700 1 -- 172.21.15.6:0/1243662155 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52f40042c0 con 0x7f530007de40 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.405+0000 7f52ff400700 1 -- 172.21.15.6:0/1243662155 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f4011ac0 con 0x7f530007de40 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 -- 172.21.15.6:0/1243662155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 msgr2=0x7f530007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 --2- 172.21.15.6:0/1243662155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530007c2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52f40098a0 tx=0x7f52f4009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 -- 172.21.15.6:0/1243662155 shutdown_connections 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 --2- 172.21.15.6:0/1243662155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530007c2a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 -- 172.21.15.6:0/1243662155 >> 172.21.15.6:0/1243662155 conn(0x7f5300102d60 msgr2=0x7f5300105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 -- 172.21.15.6:0/1243662155 shutdown_connections 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.406+0000 7f5307400700 1 -- 172.21.15.6:0/1243662155 wait complete. 2024-02-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.407+0000 7f5307400700 1 Processor -- start 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.407+0000 7f5307400700 1 -- start start 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.407+0000 7f5307400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.407+0000 7f5307400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530019dc00 con 0x7f530007de40 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.408+0000 7f5304a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.408+0000 7f5304a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45452/0 (socket says 172.21.15.6:45452) 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.408+0000 7f5304a00700 1 -- 172.21.15.6:0/130240084 learned_addr learned my addr 172.21.15.6:0/130240084 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.408+0000 7f5304a00700 1 -- 172.21.15.6:0/130240084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009550 con 0x7f530007de40 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.408+0000 7f5304a00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530019d6c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009130 tx=0x7f52f4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.409+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f4003aa0 con 0x7f530007de40 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.409+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52f4003c00 con 0x7f530007de40 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.409+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f4024800 con 0x7f530007de40 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.409+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530019de00 con 0x7f530007de40 2024-02-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.409+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530019e2a0 con 0x7f530007de40 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.410+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f52f401abd0 con 0x7f530007de40 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.410+0000 7f52fd600700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.410+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 --> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52e803bb60 con 0x7f52e8038f90 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.410+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f52f404cd50 con 0x7f530007de40 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.410+0000 7f52ffe00700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 msgr2=0x7f52e803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.410+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.612+0000 7f52ffe00700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 msgr2=0x7f52e803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:30.612+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:31.013+0000 7f52ffe00700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 msgr2=0x7f52e803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:31.013+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:31.814+0000 7f52ffe00700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 msgr2=0x7f52e803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:31.814+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:33.416+0000 7f52ffe00700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 msgr2=0x7f52e803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/1027725899 2024-02-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:33.416+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:36.559+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7f52f4024c50 con 0x7f530007de40 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:36.560+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 msgr2=0x7f52e803b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:36.560+0000 7f52fd600700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.563+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f52f404e420 con 0x7f530007de40 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.563+0000 7f52fd600700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f52e803c8b0 0x7f52e803eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.563+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52e803bb60 con 0x7f52e803c8b0 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.564+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f52e803c8b0 0x7f52e803eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.566+0000 7f52ffe00700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f52e803c8b0 0x7f52e803eca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52ec003a10 tx=0x7f52ec0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.567+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f52e803bb60 con 0x7f52e803c8b0 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.576+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f530010db90 con 0x7f52e803c8b0 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f52fd600700 1 -- 172.21.15.6:0/130240084 <== mgr.14120 v2:172.21.15.6:6800/4032437212 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f530010db90 con 0x7f52e803c8b0 2024-02-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f52e803c8b0 msgr2=0x7f52e803eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f52e803c8b0 0x7f52e803eca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52ec003a10 tx=0x7f52ec0092b0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 msgr2=0x7f530019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530019d6c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009130 tx=0x7f52f4011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 shutdown_connections 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f52e803c8b0 0x7f52e803eca0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530007de40 0x7f530019d6c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 --2- 172.21.15.6:0/130240084 >> [v2:172.21.15.6:6800/1027725899,v1:172.21.15.6:6801/1027725899] conn(0x7f52e8038f90 0x7f52e803b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.577+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 >> 172.21.15.6:0/130240084 conn(0x7f5300102d60 msgr2=0x7f5300103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.578+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 shutdown_connections 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:37.578+0000 7f5307400700 1 -- 172.21.15.6:0/130240084 wait complete. 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 5 is available 2024-02-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi006.stdout:Setting orchestrator backend to cephadm... 2024-02-26T06:16:38.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:38 smithi006 ceph-mon[31790]: [26/Feb/2024:06:16:37] ENGINE Serving on https://172.21.15.6:7150 2024-02-26T06:16:38.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:38 smithi006 ceph-mon[31790]: [26/Feb/2024:06:16:37] ENGINE Serving on http://172.21.15.6:8765 2024-02-26T06:16:38.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:38 smithi006 ceph-mon[31790]: [26/Feb/2024:06:16:37] ENGINE Bus STARTED 2024-02-26T06:16:38.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:38 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:39.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:39 smithi006 ceph-mon[31790]: mgrmap e8: x(active, since 2s) 2024-02-26T06:16:39.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:39 smithi006 ceph-mon[31790]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:39.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:39 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:39.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:39 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:39.901 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.902+0000 7f4c3b200700 1 Processor -- start 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.902+0000 7f4c3b200700 1 -- start start 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.903+0000 7f4c3b200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 0x7f4c3c10a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.903+0000 7f4c3b200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c075550 con 0x7f4c3c1080a0 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.903+0000 7f4c39e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 0x7f4c3c10a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.903+0000 7f4c39e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 0x7f4c3c10a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37348/0 (socket says 172.21.15.6:37348) 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.903+0000 7f4c39e00700 1 -- 172.21.15.6:0/1718282602 learned_addr learned my addr 172.21.15.6:0/1718282602 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.904+0000 7f4c39e00700 1 -- 172.21.15.6:0/1718282602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c10a9d0 con 0x7f4c3c1080a0 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.904+0000 7f4c39e00700 1 --2- 172.21.15.6:0/1718282602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 0x7f4c3c10a490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009e80 tx=0x7f4c2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53eb61c235122667 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:39.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.904+0000 7f4c39400700 1 -- 172.21.15.6:0/1718282602 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c2c00e070 con 0x7f4c3c1080a0 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c39400700 1 -- 172.21.15.6:0/1718282602 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2c00b8c0 con 0x7f4c3c1080a0 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c39400700 1 -- 172.21.15.6:0/1718282602 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c2c004550 con 0x7f4c3c1080a0 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c3b200700 1 -- 172.21.15.6:0/1718282602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 msgr2=0x7f4c3c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c3b200700 1 --2- 172.21.15.6:0/1718282602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 0x7f4c3c10a490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009e80 tx=0x7f4c2c00b330 comp rx=0 tx=0).stop 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c3b200700 1 -- 172.21.15.6:0/1718282602 shutdown_connections 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c3b200700 1 --2- 172.21.15.6:0/1718282602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c1080a0 0x7f4c3c10a490 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.905+0000 7f4c3b200700 1 -- 172.21.15.6:0/1718282602 >> 172.21.15.6:0/1718282602 conn(0x7f4c3c101ad0 msgr2=0x7f4c3c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.906+0000 7f4c3b200700 1 -- 172.21.15.6:0/1718282602 shutdown_connections 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.906+0000 7f4c3b200700 1 -- 172.21.15.6:0/1718282602 wait complete. 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.907+0000 7f4c3b200700 1 Processor -- start 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.907+0000 7f4c3b200700 1 -- start start 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.908+0000 7f4c3b200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 0x7f4c3c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.908+0000 7f4c3b200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c19e020 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.908+0000 7f4c39e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 0x7f4c3c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.908+0000 7f4c39e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 0x7f4c3c19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37356/0 (socket says 172.21.15.6:37356) 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.908+0000 7f4c39e00700 1 -- 172.21.15.6:0/1278009183 learned_addr learned my addr 172.21.15.6:0/1278009183 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.908+0000 7f4c39e00700 1 -- 172.21.15.6:0/1278009183 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009b30 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.909+0000 7f4c39e00700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 0x7f4c3c19dae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0041a0 tx=0x7f4c2c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.909+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c2c00e040 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.909+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c19e220 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.909+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c1a0d90 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.910+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2c00bd70 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.910+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c2c0126c0 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.911+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c10b780 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.911+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f4c2c01e450 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.911+0000 7f4c33400700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4c20038ee0 0x7f4c2003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.912+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4c2c04d410 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.912+0000 7f4c31400700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4c20038ee0 0x7f4c2003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.913+0000 7f4c31400700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4c20038ee0 0x7f4c2003b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006fd0 tx=0x7f4c24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:38.920+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c2c02fa40 con 0x7f4c3c19d6c0 2024-02-26T06:16:39.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.161+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c3c102cc0 con 0x7f4c20038ee0 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.173+0000 7f4c33400700 1 -- 172.21.15.6:0/1278009183 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4c3c102cc0 con 0x7f4c20038ee0 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4c20038ee0 msgr2=0x7f4c2003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4c20038ee0 0x7f4c2003b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006fd0 tx=0x7f4c24006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 msgr2=0x7f4c3c19dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 0x7f4c3c19dae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0041a0 tx=0x7f4c2c003d00 comp rx=0 tx=0).stop 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 shutdown_connections 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4c20038ee0 0x7f4c2003b3a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 --2- 172.21.15.6:0/1278009183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c19d6c0 0x7f4c3c19dae0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 >> 172.21.15.6:0/1278009183 conn(0x7f4c3c101ad0 msgr2=0x7f4c3c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.176+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 shutdown_connections 2024-02-26T06:16:39.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:39.177+0000 7f4c3b200700 1 -- 172.21.15.6:0/1278009183 wait complete. 2024-02-26T06:16:41.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-26T06:16:41.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.509+0000 7fe76f600700 1 Processor -- start 2024-02-26T06:16:41.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.510+0000 7fe76f600700 1 -- start start 2024-02-26T06:16:41.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.510+0000 7fe76f600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe768108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:41.444 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.510+0000 7fe76f600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe768108700 con 0x7fe768107d10 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.511+0000 7fe76cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe768108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.511+0000 7fe76cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe768108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37360/0 (socket says 172.21.15.6:37360) 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.511+0000 7fe76cc00700 1 -- 172.21.15.6:0/228461110 learned_addr learned my addr 172.21.15.6:0/228461110 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.511+0000 7fe76cc00700 1 -- 172.21.15.6:0/228461110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768108840 con 0x7fe768107d10 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.511+0000 7fe76cc00700 1 --2- 172.21.15.6:0/228461110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe768108130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0098a0 tx=0x7fe75c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bd2d72f4f46b0d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.512+0000 7fe767400700 1 -- 172.21.15.6:0/228461110 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe75c004160 con 0x7fe768107d10 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.512+0000 7fe767400700 1 -- 172.21.15.6:0/228461110 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe75c0042c0 con 0x7fe768107d10 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.512+0000 7fe767400700 1 -- 172.21.15.6:0/228461110 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe75c011ac0 con 0x7fe768107d10 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.512+0000 7fe76f600700 1 -- 172.21.15.6:0/228461110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 msgr2=0x7fe768108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.512+0000 7fe76f600700 1 --2- 172.21.15.6:0/228461110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe768108130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0098a0 tx=0x7fe75c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.513+0000 7fe76f600700 1 -- 172.21.15.6:0/228461110 shutdown_connections 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.513+0000 7fe76f600700 1 --2- 172.21.15.6:0/228461110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe768108130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.513+0000 7fe76f600700 1 -- 172.21.15.6:0/228461110 >> 172.21.15.6:0/228461110 conn(0x7fe7681013f0 msgr2=0x7fe768103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:41.445 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.513+0000 7fe76f600700 1 -- 172.21.15.6:0/228461110 shutdown_connections 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.513+0000 7fe76f600700 1 -- 172.21.15.6:0/228461110 wait complete. 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76f600700 1 Processor -- start 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76f600700 1 -- start start 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76f600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe76819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76f600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76819dcf0 con 0x7fe768107d10 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe76819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe76819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37374/0 (socket says 172.21.15.6:37374) 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.514+0000 7fe76cc00700 1 -- 172.21.15.6:0/256354322 learned_addr learned my addr 172.21.15.6:0/256354322 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.515+0000 7fe76cc00700 1 -- 172.21.15.6:0/256354322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009550 con 0x7fe768107d10 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.515+0000 7fe76cc00700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe76819d7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe75c011a70 tx=0x7fe75c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.515+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe75c0039c0 con 0x7fe768107d10 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.515+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe75c003b20 con 0x7fe768107d10 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.515+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe75c024610 con 0x7fe768107d10 2024-02-26T06:16:41.446 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.515+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76819def0 con 0x7fe768107d10 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.516+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76819e390 con 0x7fe768107d10 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.517+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7681064e0 con 0x7fe768107d10 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.517+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fe75c01aa10 con 0x7fe768107d10 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.517+0000 7fe765600700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fe750038f30 0x7fe75003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.517+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe75c04cfb0 con 0x7fe768107d10 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.518+0000 7fe767e00700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fe750038f30 0x7fe75003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.518+0000 7fe767e00700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fe750038f30 0x7fe75003b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe754006fd0 tx=0x7fe754006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.523+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe75c01a430 con 0x7fe768107d10 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.746+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7680620b0 con 0x7fe750038f30 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.748+0000 7fe765600700 1 -- 172.21.15.6:0/256354322 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe7680620b0 con 0x7fe750038f30 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.750+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fe750038f30 msgr2=0x7fe75003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.750+0000 7fe76f600700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fe750038f30 0x7fe75003b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe754006fd0 tx=0x7fe754006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.750+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 msgr2=0x7fe76819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.750+0000 7fe76f600700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe76819d7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe75c011a70 tx=0x7fe75c011b50 comp rx=0 tx=0).stop 2024-02-26T06:16:41.447 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.751+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 shutdown_connections 2024-02-26T06:16:41.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.751+0000 7fe76f600700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fe750038f30 0x7fe75003b3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:41.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.751+0000 7fe76f600700 1 --2- 172.21.15.6:0/256354322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe768107d10 0x7fe76819d7b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:41.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.751+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 >> 172.21.15.6:0/256354322 conn(0x7fe7681013f0 msgr2=0x7fe768193fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:41.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.751+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 shutdown_connections 2024-02-26T06:16:41.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:40.751+0000 7fe76f600700 1 -- 172.21.15.6:0/256354322 wait complete. 2024-02-26T06:16:41.448 INFO:teuthology.orchestra.run.smithi006.stdout:Generating ssh key... 2024-02-26T06:16:41.762 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:41 smithi006 ceph-mon[31790]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:42.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-mon[31790]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:42.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-mon[31790]: Generating ssh key... 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: Generating public/private rsa key pair. 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: Your identification has been saved in /tmp/tmp_k5_l0m2/key. 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: Your public key has been saved in /tmp/tmp_k5_l0m2/key.pub. 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: The key fingerprint is: 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: SHA256:slLEwBQyQ3fXWRxA3gWBN1/0hG9kIS6e1xCFYjV5BOY ceph-5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: The key's randomart image is: 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: +---[RSA 3072]----+ 2024-02-26T06:16:42.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | .=o=.. .oo**@B*+| 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | =.+ . .++B+=o=| 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | o oooE.=.| 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | . . o + o| 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | o S o . o | 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | . o . | 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | . . | 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | . | 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: | | 2024-02-26T06:16:42.923 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:16:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: +----[SHA256]-----+ 2024-02-26T06:16:43.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.087+0000 7fcc72400700 1 Processor -- start 2024-02-26T06:16:43.285 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.088+0000 7fcc72400700 1 -- start start 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.088+0000 7fcc72400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.088+0000 7fcc72400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c108170 con 0x7fcc6c107780 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.088+0000 7fcc6b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.088+0000 7fcc6b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37390/0 (socket says 172.21.15.6:37390) 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.088+0000 7fcc6b400700 1 -- 172.21.15.6:0/750982392 learned_addr learned my addr 172.21.15.6:0/750982392 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.089+0000 7fcc6b400700 1 -- 172.21.15.6:0/750982392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c108980 con 0x7fcc6c107780 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.089+0000 7fcc6b400700 1 --2- 172.21.15.6:0/750982392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c107ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009e80 tx=0x7fcc5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c9bc3f51a6487a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.089+0000 7fcc6a000700 1 -- 172.21.15.6:0/750982392 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc5c00e070 con 0x7fcc6c107780 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.089+0000 7fcc6a000700 1 -- 172.21.15.6:0/750982392 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc5c00b8c0 con 0x7fcc6c107780 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.090+0000 7fcc6a000700 1 -- 172.21.15.6:0/750982392 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc5c004550 con 0x7fcc6c107780 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.090+0000 7fcc72400700 1 -- 172.21.15.6:0/750982392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 msgr2=0x7fcc6c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.090+0000 7fcc72400700 1 --2- 172.21.15.6:0/750982392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c107ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009e80 tx=0x7fcc5c00b330 comp rx=0 tx=0).stop 2024-02-26T06:16:43.286 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.090+0000 7fcc72400700 1 -- 172.21.15.6:0/750982392 shutdown_connections 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.090+0000 7fcc72400700 1 --2- 172.21.15.6:0/750982392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c107ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.090+0000 7fcc72400700 1 -- 172.21.15.6:0/750982392 >> 172.21.15.6:0/750982392 conn(0x7fcc6c102d80 msgr2=0x7fcc6c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.091+0000 7fcc72400700 1 -- 172.21.15.6:0/750982392 shutdown_connections 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.091+0000 7fcc72400700 1 -- 172.21.15.6:0/750982392 wait complete. 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc72400700 1 Processor -- start 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc72400700 1 -- start start 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc72400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc72400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c19dc40 con 0x7fcc6c107780 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc6b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc6b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37406/0 (socket says 172.21.15.6:37406) 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.092+0000 7fcc6b400700 1 -- 172.21.15.6:0/1629394845 learned_addr learned my addr 172.21.15.6:0/1629394845 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.093+0000 7fcc6b400700 1 -- 172.21.15.6:0/1629394845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c009b30 con 0x7fcc6c107780 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.093+0000 7fcc6b400700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c19d700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009fd0 tx=0x7fcc5c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.093+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc5c00e040 con 0x7fcc6c107780 2024-02-26T06:16:43.287 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.093+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc5c0128f0 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.093+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc5c01c770 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.093+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c19de40 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.094+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c19e2c0 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.095+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fcc5c012a60 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.095+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c10bbb0 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.095+0000 7fcc63e00700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fcc50038ee0 0x7fcc5003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.095+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc5c04cfd0 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.095+0000 7fcc6aa00700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fcc50038ee0 0x7fcc5003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.096+0000 7fcc6aa00700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fcc50038ee0 0x7fcc5003b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006fd0 tx=0x7fcc58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.101+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc5c024c00 con 0x7fcc6c107780 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.326+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc6c103e60 con 0x7fcc50038ee0 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.712+0000 7fcc63e00700 1 -- 172.21.15.6:0/1629394845 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc6c103e60 con 0x7fcc50038ee0 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fcc50038ee0 msgr2=0x7fcc5003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:43.288 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fcc50038ee0 0x7fcc5003b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006fd0 tx=0x7fcc58006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 msgr2=0x7fcc6c19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c19d700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009fd0 tx=0x7fcc5c003d00 comp rx=0 tx=0).stop 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 shutdown_connections 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fcc50038ee0 0x7fcc5003b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 --2- 172.21.15.6:0/1629394845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc6c107780 0x7fcc6c19d700 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 >> 172.21.15.6:0/1629394845 conn(0x7fcc6c102d80 msgr2=0x7fcc6c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 shutdown_connections 2024-02-26T06:16:43.289 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:42.715+0000 7fcc72400700 1 -- 172.21.15.6:0/1629394845 wait complete. 2024-02-26T06:16:43.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:43 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:43.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:43 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:44.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:44 smithi006 ceph-mon[31790]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:44.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:16:44.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.890+0000 7f7b72c00700 1 Processor -- start 2024-02-26T06:16:44.867 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.890+0000 7f7b72c00700 1 -- start start 2024-02-26T06:16:44.867 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.891+0000 7f7b72c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c081bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:44.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.891+0000 7f7b72c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c082180 con 0x7f7b6c081790 2024-02-26T06:16:44.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.891+0000 7f7b6be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c081bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:44.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.891+0000 7f7b6be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c081bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37418/0 (socket says 172.21.15.6:37418) 2024-02-26T06:16:44.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.891+0000 7f7b6be00700 1 -- 172.21.15.6:0/4143734426 learned_addr learned my addr 172.21.15.6:0/4143734426 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:44.869 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.892+0000 7f7b6be00700 1 -- 172.21.15.6:0/4143734426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c10ef60 con 0x7f7b6c081790 2024-02-26T06:16:44.869 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.892+0000 7f7b6be00700 1 --2- 172.21.15.6:0/4143734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c081bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0098a0 tx=0x7f7b5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f015efe0a08ad0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:44.869 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.893+0000 7f7b6b400700 1 -- 172.21.15.6:0/4143734426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b5c004160 con 0x7f7b6c081790 2024-02-26T06:16:44.869 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.893+0000 7f7b6b400700 1 -- 172.21.15.6:0/4143734426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b5c0042c0 con 0x7f7b6c081790 2024-02-26T06:16:44.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.893+0000 7f7b6b400700 1 -- 172.21.15.6:0/4143734426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b5c011ac0 con 0x7f7b6c081790 2024-02-26T06:16:44.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.893+0000 7f7b72c00700 1 -- 172.21.15.6:0/4143734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 msgr2=0x7f7b6c081bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:44.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.893+0000 7f7b72c00700 1 --2- 172.21.15.6:0/4143734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c081bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0098a0 tx=0x7f7b5c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:44.870 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.894+0000 7f7b72c00700 1 -- 172.21.15.6:0/4143734426 shutdown_connections 2024-02-26T06:16:44.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.894+0000 7f7b72c00700 1 --2- 172.21.15.6:0/4143734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c081bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:44.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.894+0000 7f7b72c00700 1 -- 172.21.15.6:0/4143734426 >> 172.21.15.6:0/4143734426 conn(0x7f7b6c07cd50 msgr2=0x7f7b6c07f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:44.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.894+0000 7f7b72c00700 1 -- 172.21.15.6:0/4143734426 shutdown_connections 2024-02-26T06:16:44.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.894+0000 7f7b72c00700 1 -- 172.21.15.6:0/4143734426 wait complete. 2024-02-26T06:16:44.871 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.895+0000 7f7b72c00700 1 Processor -- start 2024-02-26T06:16:44.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.895+0000 7f7b72c00700 1 -- start start 2024-02-26T06:16:44.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.895+0000 7f7b72c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:44.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.895+0000 7f7b72c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c1a2080 con 0x7f7b6c081790 2024-02-26T06:16:44.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.895+0000 7f7b6be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:44.872 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.895+0000 7f7b6be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c1a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37424/0 (socket says 172.21.15.6:37424) 2024-02-26T06:16:44.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.896+0000 7f7b6be00700 1 -- 172.21.15.6:0/1057266008 learned_addr learned my addr 172.21.15.6:0/1057266008 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:44.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.896+0000 7f7b6be00700 1 -- 172.21.15.6:0/1057266008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009550 con 0x7f7b6c081790 2024-02-26T06:16:44.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.896+0000 7f7b6be00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c1a1b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000c00 tx=0x7f7b5c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:44.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.896+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b5c003aa0 con 0x7f7b6c081790 2024-02-26T06:16:44.873 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.897+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b5c003c00 con 0x7f7b6c081790 2024-02-26T06:16:44.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.897+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c1a2280 con 0x7f7b6c081790 2024-02-26T06:16:44.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.897+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b5c0248a0 con 0x7f7b6c081790 2024-02-26T06:16:44.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.897+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c1a2720 con 0x7f7b6c081790 2024-02-26T06:16:44.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.898+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7b5c01ac60 con 0x7f7b6c081790 2024-02-26T06:16:44.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.898+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c112190 con 0x7f7b6c081790 2024-02-26T06:16:44.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.898+0000 7f7b69600700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f7b50038f30 0x7f7b5003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:44.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.898+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b5c04d060 con 0x7f7b6c081790 2024-02-26T06:16:44.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.899+0000 7f7b63e00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f7b50038f30 0x7f7b5003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:44.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.899+0000 7f7b63e00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f7b50038f30 0x7f7b5003b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006fd0 tx=0x7f7b54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:44.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:43.904+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b5c018530 con 0x7f7b6c081790 2024-02-26T06:16:44.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.145+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b6c07df50 con 0x7f7b50038f30 2024-02-26T06:16:44.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.147+0000 7f7b69600700 1 -- 172.21.15.6:0/1057266008 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7b6c07df50 con 0x7f7b50038f30 2024-02-26T06:16:44.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f7b50038f30 msgr2=0x7f7b5003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:44.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f7b50038f30 0x7f7b5003b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006fd0 tx=0x7f7b54006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:44.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 msgr2=0x7f7b6c1a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:44.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c1a1b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000c00 tx=0x7f7b5c011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:44.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 shutdown_connections 2024-02-26T06:16:44.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f7b50038f30 0x7f7b5003b3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:44.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 --2- 172.21.15.6:0/1057266008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c081790 0x7f7b6c1a1b40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:44.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.150+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 >> 172.21.15.6:0/1057266008 conn(0x7f7b6c07cd50 msgr2=0x7f7b6c07d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:44.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.151+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 shutdown_connections 2024-02-26T06:16:44.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:44.151+0000 7f7b72c00700 1 -- 172.21.15.6:0/1057266008 wait complete. 2024-02-26T06:16:44.879 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:16:44.880 INFO:teuthology.orchestra.run.smithi006.stdout:Adding key to root@localhost authorized_keys... 2024-02-26T06:16:44.881 INFO:teuthology.orchestra.run.smithi006.stdout:Adding host smithi006... 2024-02-26T06:16:46.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:46 smithi006 ceph-mon[31790]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:47.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:47 smithi006 ceph-mon[31790]: Deploying cephadm binary to smithi006 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Added host 'smithi006' with addr '172.21.15.6' 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.483+0000 7ffbc0e00700 1 Processor -- start 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.483+0000 7ffbc0e00700 1 -- start start 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.484+0000 7ffbc0e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc10a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.484+0000 7ffbc0e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0755d0 con 0x7ffbbc1081d0 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.484+0000 7ffbb9e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc10a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.484+0000 7ffbb9e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc10a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37434/0 (socket says 172.21.15.6:37434) 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.484+0000 7ffbb9e00700 1 -- 172.21.15.6:0/1996062261 learned_addr learned my addr 172.21.15.6:0/1996062261 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.485+0000 7ffbb9e00700 1 -- 172.21.15.6:0/1996062261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc10ab00 con 0x7ffbbc1081d0 2024-02-26T06:16:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.485+0000 7ffbb9e00700 1 --2- 172.21.15.6:0/1996062261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc10a5c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0098a0 tx=0x7ffbac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343f8feb210593d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.485+0000 7ffbb8a00700 1 -- 172.21.15.6:0/1996062261 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac004160 con 0x7ffbbc1081d0 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.485+0000 7ffbb8a00700 1 -- 172.21.15.6:0/1996062261 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbac0042c0 con 0x7ffbbc1081d0 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.485+0000 7ffbb8a00700 1 -- 172.21.15.6:0/1996062261 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac011ac0 con 0x7ffbbc1081d0 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1996062261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 msgr2=0x7ffbbc10a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 --2- 172.21.15.6:0/1996062261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc10a5c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0098a0 tx=0x7ffbac009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1996062261 shutdown_connections 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 --2- 172.21.15.6:0/1996062261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc10a5c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1996062261 >> 172.21.15.6:0/1996062261 conn(0x7ffbbc101c20 msgr2=0x7ffbbc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1996062261 shutdown_connections 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.486+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1996062261 wait complete. 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.487+0000 7ffbc0e00700 1 Processor -- start 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.488+0000 7ffbc0e00700 1 -- start start 2024-02-26T06:16:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.488+0000 7ffbc0e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.488+0000 7ffbc0e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc107f90 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.489+0000 7ffbb9e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.489+0000 7ffbb9e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37448/0 (socket says 172.21.15.6:37448) 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.489+0000 7ffbb9e00700 1 -- 172.21.15.6:0/1143734684 learned_addr learned my addr 172.21.15.6:0/1143734684 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.489+0000 7ffbb9e00700 1 -- 172.21.15.6:0/1143734684 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac009550 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.489+0000 7ffbb9e00700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc107a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009fb0 tx=0x7ffbac011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.490+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac0038b0 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.490+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbac003a10 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.490+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc106180 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.490+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac0195e0 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.490+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc106620 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.491+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ffbac01a9f0 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.491+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc10b960 con 0x7ffbbc1081d0 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.491+0000 7ffbb2a00700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ffba0038e90 0x7ffba003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.491+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffbac04d200 con 0x7ffbbc1081d0 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.492+0000 7ffbb9400700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ffba0038e90 0x7ffba003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.492+0000 7ffbb9400700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ffba0038e90 0x7ffba003b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006fd0 tx=0x7ffba8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.498+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffbac003b80 con 0x7ffbbc1081d0 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:45.728+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbbc106d10 con 0x7ffba0038e90 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.449+0000 7ffbb2a00700 1 -- 172.21.15.6:0/1143734684 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ffbbc106d10 con 0x7ffba0038e90 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.453+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ffba0038e90 msgr2=0x7ffba003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.453+0000 7ffbc0e00700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ffba0038e90 0x7ffba003b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006fd0 tx=0x7ffba8006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 msgr2=0x7ffbbc107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc107a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009fb0 tx=0x7ffbac011b20 comp rx=0 tx=0).stop 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 shutdown_connections 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ffba0038e90 0x7ffba003b350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 --2- 172.21.15.6:0/1143734684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbbc1081d0 0x7ffbbc107a50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 >> 172.21.15.6:0/1143734684 conn(0x7ffbbc101c20 msgr2=0x7ffbbc102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 shutdown_connections 2024-02-26T06:16:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:52.454+0000 7ffbc0e00700 1 -- 172.21.15.6:0/1143734684 wait complete. 2024-02-26T06:16:53.175 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mon service... 2024-02-26T06:16:53.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:53 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:53.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:53 smithi006 ceph-mon[31790]: Added host smithi006 2024-02-26T06:16:53.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:53 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:16:55.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:55 smithi006 ceph-mon[31790]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:55.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:55 smithi006 ceph-mon[31790]: Saving service mon spec with placement count:5 2024-02-26T06:16:55.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:55 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-26T06:16:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.851+0000 7f5022c00700 1 Processor -- start 2024-02-26T06:16:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.851+0000 7f5022c00700 1 -- start start 2024-02-26T06:16:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.852+0000 7f5022c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.852+0000 7f5022c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c10a360 con 0x7f501c109970 2024-02-26T06:16:55.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.852+0000 7f501be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.852+0000 7f501be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46156/0 (socket says 172.21.15.6:46156) 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.852+0000 7f501be00700 1 -- 172.21.15.6:0/2117157514 learned_addr learned my addr 172.21.15.6:0/2117157514 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.853+0000 7f501be00700 1 -- 172.21.15.6:0/2117157514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c10ab70 con 0x7f501c109970 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.853+0000 7f501be00700 1 --2- 172.21.15.6:0/2117157514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c109d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f500c0098a0 tx=0x7f500c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15f72d11d5bed23 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.853+0000 7f501aa00700 1 -- 172.21.15.6:0/2117157514 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f500c004160 con 0x7f501c109970 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f501aa00700 1 -- 172.21.15.6:0/2117157514 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f500c0042c0 con 0x7f501c109970 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f501aa00700 1 -- 172.21.15.6:0/2117157514 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f500c011ac0 con 0x7f501c109970 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 -- 172.21.15.6:0/2117157514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 msgr2=0x7f501c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 --2- 172.21.15.6:0/2117157514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c109d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f500c0098a0 tx=0x7f500c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 -- 172.21.15.6:0/2117157514 shutdown_connections 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 --2- 172.21.15.6:0/2117157514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c109d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 -- 172.21.15.6:0/2117157514 >> 172.21.15.6:0/2117157514 conn(0x7f501c07cd80 msgr2=0x7f501c07d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 -- 172.21.15.6:0/2117157514 shutdown_connections 2024-02-26T06:16:55.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.854+0000 7f5022c00700 1 -- 172.21.15.6:0/2117157514 wait complete. 2024-02-26T06:16:55.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.855+0000 7f5022c00700 1 Processor -- start 2024-02-26T06:16:55.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.856+0000 7f5022c00700 1 -- start start 2024-02-26T06:16:55.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.856+0000 7f5022c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:55.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.856+0000 7f5022c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19dc70 con 0x7f501c109970 2024-02-26T06:16:55.837 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.856+0000 7f501be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.856+0000 7f501be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46164/0 (socket says 172.21.15.6:46164) 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.856+0000 7f501be00700 1 -- 172.21.15.6:0/2131151890 learned_addr learned my addr 172.21.15.6:0/2131151890 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f501be00700 1 -- 172.21.15.6:0/2131151890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c009550 con 0x7f501c109970 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f501be00700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c19d730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f500c000c00 tx=0x7f500c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f500c003aa0 con 0x7f501c109970 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f500c003c00 con 0x7f501c109970 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c19de70 con 0x7f501c109970 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f500c0248a0 con 0x7f501c109970 2024-02-26T06:16:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.857+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c19e310 con 0x7f501c109970 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.859+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f500c02d3f0 con 0x7f501c109970 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.859+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c10dda0 con 0x7f501c109970 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.859+0000 7f5018c00700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f5000038f30 0x7f500003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.859+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f500c04d190 con 0x7f501c109970 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.859+0000 7f501b400700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f5000038f30 0x7f500003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.860+0000 7f501b400700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f5000038f30 0x7f500003b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5008006fd0 tx=0x7f5008006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:53.865+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f500c010a70 con 0x7f501c109970 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.117+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f501c107b80 con 0x7f5000038f30 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.124+0000 7f5018c00700 1 -- 172.21.15.6:0/2131151890 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f501c107b80 con 0x7f5000038f30 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f5000038f30 msgr2=0x7f500003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f5000038f30 0x7f500003b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5008006fd0 tx=0x7f5008006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 msgr2=0x7f501c19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c19d730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f500c000c00 tx=0x7f500c011b40 comp rx=0 tx=0).stop 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 shutdown_connections 2024-02-26T06:16:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f5000038f30 0x7f500003b3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:55.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 --2- 172.21.15.6:0/2131151890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f501c109970 0x7f501c19d730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:55.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.127+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 >> 172.21.15.6:0/2131151890 conn(0x7f501c07cd80 msgr2=0x7f501c107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:55.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.128+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 shutdown_connections 2024-02-26T06:16:55.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:54.128+0000 7f5022c00700 1 -- 172.21.15.6:0/2131151890 wait complete. 2024-02-26T06:16:55.842 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mgr service... 2024-02-26T06:16:57.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:57 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:57.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:57 smithi006 ceph-mon[31790]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:16:57.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:57 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:57.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:57 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.545+0000 7ffa16a00700 1 Processor -- start 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.545+0000 7ffa16a00700 1 -- start start 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.546+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.546+0000 7ffa16a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1007c8d0 con 0x7ffa1007dea0 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.546+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:57.647 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.546+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46174/0 (socket says 172.21.15.6:46174) 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.546+0000 7ffa0fe00700 1 -- 172.21.15.6:0/2299322923 learned_addr learned my addr 172.21.15.6:0/2299322923 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.547+0000 7ffa0fe00700 1 -- 172.21.15.6:0/2299322923 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1007ca10 con 0x7ffa1007dea0 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.547+0000 7ffa0fe00700 1 --2- 172.21.15.6:0/2299322923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1007c300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa000098a0 tx=0x7ffa00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493a2ba46f271cc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.547+0000 7ffa0ea00700 1 -- 172.21.15.6:0/2299322923 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa00004160 con 0x7ffa1007dea0 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.547+0000 7ffa0ea00700 1 -- 172.21.15.6:0/2299322923 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa000042c0 con 0x7ffa1007dea0 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.547+0000 7ffa0ea00700 1 -- 172.21.15.6:0/2299322923 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa00011a80 con 0x7ffa1007dea0 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 -- 172.21.15.6:0/2299322923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 msgr2=0x7ffa1007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 --2- 172.21.15.6:0/2299322923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1007c300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa000098a0 tx=0x7ffa00009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 -- 172.21.15.6:0/2299322923 shutdown_connections 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 --2- 172.21.15.6:0/2299322923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1007c300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 -- 172.21.15.6:0/2299322923 >> 172.21.15.6:0/2299322923 conn(0x7ffa10102d00 msgr2=0x7ffa10105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 -- 172.21.15.6:0/2299322923 shutdown_connections 2024-02-26T06:16:57.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.548+0000 7ffa16a00700 1 -- 172.21.15.6:0/2299322923 wait complete. 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.549+0000 7ffa16a00700 1 Processor -- start 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.549+0000 7ffa16a00700 1 -- start start 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.550+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.550+0000 7ffa16a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019dc40 con 0x7ffa1007dea0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.550+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.550+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46186/0 (socket says 172.21.15.6:46186) 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.550+0000 7ffa0fe00700 1 -- 172.21.15.6:0/3286098052 learned_addr learned my addr 172.21.15.6:0/3286098052 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa0fe00700 1 -- 172.21.15.6:0/3286098052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00009550 con 0x7ffa1007dea0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa0fe00700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1019d700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa00000c00 tx=0x7ffa00011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa00003aa0 con 0x7ffa1007dea0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1019de40 con 0x7ffa1007dea0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa00003c00 con 0x7ffa1007dea0 2024-02-26T06:16:57.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa00024930 con 0x7ffa1007dea0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.551+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1019e2e0 con 0x7ffa1007dea0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.552+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1010dbc0 con 0x7ffa1007dea0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.553+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ffa0001aca0 con 0x7ffa1007dea0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.553+0000 7ffa0cc00700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ff9f4038ee0 0x7ff9f403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.553+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffa0004d1c0 con 0x7ffa1007dea0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.553+0000 7ffa0f400700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ff9f4038ee0 0x7ff9f403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.554+0000 7ffa0f400700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ff9f4038ee0 0x7ff9f403b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006fd0 tx=0x7ff9fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.563+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa00018710 con 0x7ffa1007dea0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.812+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ffa10103f00 con 0x7ff9f4038ee0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.820+0000 7ffa0cc00700 1 -- 172.21.15.6:0/3286098052 <== mgr.14120 v2:172.21.15.6:6800/4032437212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ffa10103f00 con 0x7ff9f4038ee0 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ff9f4038ee0 msgr2=0x7ff9f403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ff9f4038ee0 0x7ff9f403b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006fd0 tx=0x7ff9fc006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:57.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 msgr2=0x7ffa1019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1019d700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa00000c00 tx=0x7ffa00011b00 comp rx=0 tx=0).stop 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 shutdown_connections 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7ff9f4038ee0 0x7ff9f403b3a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 --2- 172.21.15.6:0/3286098052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1007dea0 0x7ffa1019d700 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 >> 172.21.15.6:0/3286098052 conn(0x7ffa10102d00 msgr2=0x7ffa101037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.823+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 shutdown_connections 2024-02-26T06:16:57.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:56.824+0000 7ffa16a00700 1 -- 172.21.15.6:0/3286098052 wait complete. 2024-02-26T06:16:58.263 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:58 smithi006 ceph-mon[31790]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:16:58.264 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:58 smithi006 ceph-mon[31790]: Saving service mgr spec with placement count:2 2024-02-26T06:16:59.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/881695245' entity='client.admin' 2024-02-26T06:16:59.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:16:59 smithi006 ceph-mon[31790]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:16:59.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.263+0000 7f4559400700 1 Processor -- start 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.263+0000 7f4559400700 1 -- start start 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.264+0000 7f4559400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f4554107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.264+0000 7f4559400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4554108160 con 0x7f4554107770 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.264+0000 7f4552800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f4554107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.264+0000 7f4552800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f4554107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34064/0 (socket says 172.21.15.6:34064) 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.264+0000 7f4552800700 1 -- 172.21.15.6:0/1004025322 learned_addr learned my addr 172.21.15.6:0/1004025322 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.265+0000 7f4552800700 1 -- 172.21.15.6:0/1004025322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554108970 con 0x7f4554107770 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.265+0000 7f4552800700 1 --2- 172.21.15.6:0/1004025322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f4554107b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45440098a0 tx=0x7f4544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc000a20dd6dce3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.265+0000 7f4551400700 1 -- 172.21.15.6:0/1004025322 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4544004160 con 0x7f4554107770 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.265+0000 7f4551400700 1 -- 172.21.15.6:0/1004025322 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45440042c0 con 0x7f4554107770 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.265+0000 7f4551400700 1 -- 172.21.15.6:0/1004025322 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4544011a80 con 0x7f4554107770 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.266+0000 7f4559400700 1 -- 172.21.15.6:0/1004025322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 msgr2=0x7f4554107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:59.761 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.266+0000 7f4559400700 1 --2- 172.21.15.6:0/1004025322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f4554107b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45440098a0 tx=0x7f4544009bb0 comp rx=0 tx=0).stop 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.266+0000 7f4559400700 1 -- 172.21.15.6:0/1004025322 shutdown_connections 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.266+0000 7f4559400700 1 --2- 172.21.15.6:0/1004025322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f4554107b90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.266+0000 7f4559400700 1 -- 172.21.15.6:0/1004025322 >> 172.21.15.6:0/1004025322 conn(0x7f4554102d50 msgr2=0x7f4554105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.267+0000 7f4559400700 1 -- 172.21.15.6:0/1004025322 shutdown_connections 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.267+0000 7f4559400700 1 -- 172.21.15.6:0/1004025322 wait complete. 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.268+0000 7f4559400700 1 Processor -- start 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.268+0000 7f4559400700 1 -- start start 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.268+0000 7f4559400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f455407ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.268+0000 7f4559400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455407cf80 con 0x7f4554107770 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.269+0000 7f4552800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f455407ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.269+0000 7f4552800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f455407ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34066/0 (socket says 172.21.15.6:34066) 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.269+0000 7f4552800700 1 -- 172.21.15.6:0/881695245 learned_addr learned my addr 172.21.15.6:0/881695245 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.269+0000 7f4552800700 1 -- 172.21.15.6:0/881695245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544009550 con 0x7f4554107770 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.269+0000 7f4552800700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f455407ca40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:59.762 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.270+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45440039e0 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.270+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4544003b40 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.270+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455407b170 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.270+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455407b610 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.270+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4544024820 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.271+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f454401a430 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.272+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455410bba0 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.272+0000 7f454b400700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4538038ee0 0x7f453803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.272+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f454404d100 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.272+0000 7f4551e00700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4538038ee0 0x7f453803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.273+0000 7f4551e00700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4538038ee0 0x7f453803b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4540006fd0 tx=0x7f4540006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.278+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f454401ab80 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.511+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f455410bdb0 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.515+0000 7f454b400700 1 -- 172.21.15.6:0/881695245 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f454402cbe0 con 0x7f4554107770 2024-02-26T06:16:59.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4538038ee0 msgr2=0x7f453803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4538038ee0 0x7f453803b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4540006fd0 tx=0x7f4540006e40 comp rx=0 tx=0).stop 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 msgr2=0x7f455407ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f455407ca40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544011b00 comp rx=0 tx=0).stop 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 shutdown_connections 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f4538038ee0 0x7f453803b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 --2- 172.21.15.6:0/881695245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4554107770 0x7f455407ca40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.518+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 >> 172.21.15.6:0/881695245 conn(0x7f4554102d50 msgr2=0x7f45541037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.519+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 shutdown_connections 2024-02-26T06:16:59.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:16:58.519+0000 7f4559400700 1 -- 172.21.15.6:0/881695245 wait complete. 2024-02-26T06:17:00.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:00 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:17:01.334 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.345+0000 7fa4ab000700 1 Processor -- start 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.345+0000 7fa4ab000700 1 -- start start 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.345+0000 7fa4ab000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a4107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.345+0000 7fa4ab000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4108170 con 0x7fa4a4107780 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.346+0000 7fa4a3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a4107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.346+0000 7fa4a3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a4107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34078/0 (socket says 172.21.15.6:34078) 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.346+0000 7fa4a3e00700 1 -- 172.21.15.6:0/303964601 learned_addr learned my addr 172.21.15.6:0/303964601 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.346+0000 7fa4a3e00700 1 -- 172.21.15.6:0/303964601 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4108980 con 0x7fa4a4107780 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.347+0000 7fa4a3e00700 1 --2- 172.21.15.6:0/303964601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a4107ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4940098a0 tx=0x7fa494009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd213b754d1ff5a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.347+0000 7fa4a2a00700 1 -- 172.21.15.6:0/303964601 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa494004160 con 0x7fa4a4107780 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.347+0000 7fa4a2a00700 1 -- 172.21.15.6:0/303964601 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4940042c0 con 0x7fa4a4107780 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.347+0000 7fa4a2a00700 1 -- 172.21.15.6:0/303964601 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa494011a80 con 0x7fa4a4107780 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.348+0000 7fa4ab000700 1 -- 172.21.15.6:0/303964601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 msgr2=0x7fa4a4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:01.335 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.348+0000 7fa4ab000700 1 --2- 172.21.15.6:0/303964601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a4107ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4940098a0 tx=0x7fa494009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.348+0000 7fa4ab000700 1 -- 172.21.15.6:0/303964601 shutdown_connections 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.348+0000 7fa4ab000700 1 --2- 172.21.15.6:0/303964601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a4107ba0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.348+0000 7fa4ab000700 1 -- 172.21.15.6:0/303964601 >> 172.21.15.6:0/303964601 conn(0x7fa4a4102d80 msgr2=0x7fa4a41051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.349+0000 7fa4ab000700 1 -- 172.21.15.6:0/303964601 shutdown_connections 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.349+0000 7fa4ab000700 1 -- 172.21.15.6:0/303964601 wait complete. 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.350+0000 7fa4ab000700 1 Processor -- start 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.350+0000 7fa4ab000700 1 -- start start 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.351+0000 7fa4ab000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.351+0000 7fa4ab000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a419dce0 con 0x7fa4a4107780 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.351+0000 7fa4a3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.351+0000 7fa4a3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34082/0 (socket says 172.21.15.6:34082) 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.351+0000 7fa4a3e00700 1 -- 172.21.15.6:0/1017997061 learned_addr learned my addr 172.21.15.6:0/1017997061 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.352+0000 7fa4a3e00700 1 -- 172.21.15.6:0/1017997061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009550 con 0x7fa4a4107780 2024-02-26T06:17:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.352+0000 7fa4a3e00700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a419d7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa494000c00 tx=0x7fa494011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.352+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4940039e0 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.352+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa494003b40 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.352+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a419dee0 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.352+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa494024820 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.353+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a419e380 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.354+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fa49401ab80 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.354+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a410bbb0 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.354+0000 7fa4a0c00700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fa488038f30 0x7fa48803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.354+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa49404cf40 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.355+0000 7fa4a3400700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fa488038f30 0x7fa48803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.355+0000 7fa4a3400700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fa488038f30 0x7fa48803b3f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.360+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa49401a950 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.631+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa4a41a0cc0 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.635+0000 7fa4a0c00700 1 -- 172.21.15.6:0/1017997061 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa494018350 con 0x7fa4a4107780 2024-02-26T06:17:01.337 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.639+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fa488038f30 msgr2=0x7fa48803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.639+0000 7fa4ab000700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fa488038f30 0x7fa48803b3f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.639+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 msgr2=0x7fa4a419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.639+0000 7fa4ab000700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a419d7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa494000c00 tx=0x7fa494011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.640+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 shutdown_connections 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.640+0000 7fa4ab000700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7fa488038f30 0x7fa48803b3f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.640+0000 7fa4ab000700 1 --2- 172.21.15.6:0/1017997061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4a4107780 0x7fa4a419d7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.640+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 >> 172.21.15.6:0/1017997061 conn(0x7fa4a4102d80 msgr2=0x7fa4a4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.640+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 shutdown_connections 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:00.640+0000 7fa4ab000700 1 -- 172.21.15.6:0/1017997061 wait complete. 2024-02-26T06:17:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling the dashboard module... 2024-02-26T06:17:01.642 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:01 smithi006 ceph-mon[31790]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:01.643 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1017997061' entity='client.admin' 2024-02-26T06:17:03.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:02 smithi006 ceph-mon[31790]: from='mgr.14120 172.21.15.6:0/3325011013' entity='mgr.x' 2024-02-26T06:17:03.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:02 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1601999323' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-26T06:17:03.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:02 smithi006 ceph-mon[31790]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:03.173 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: ignoring --setuser ceph since I am not root 2024-02-26T06:17:03.173 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: ignoring --setgroup ceph since I am not root 2024-02-26T06:17:03.478 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.975+0000 7f962e400700 1 Processor -- start 2024-02-26T06:17:03.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.975+0000 7f962e400700 1 -- start start 2024-02-26T06:17:03.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.975+0000 7f962e400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f9628106480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:03.479 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.975+0000 7f962e400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962807be10 con 0x7f9628106060 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.976+0000 7f9627400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f9628106480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.976+0000 7f9627400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f9628106480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34094/0 (socket says 172.21.15.6:34094) 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.976+0000 7f9627400700 1 -- 172.21.15.6:0/1531510813 learned_addr learned my addr 172.21.15.6:0/1531510813 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.976+0000 7f9627400700 1 -- 172.21.15.6:0/1531510813 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96281069c0 con 0x7f9628106060 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.977+0000 7f9627400700 1 --2- 172.21.15.6:0/1531510813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f9628106480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96180098a0 tx=0x7f9618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9815e40aaa07f3ba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.977+0000 7f9626000700 1 -- 172.21.15.6:0/1531510813 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9618004160 con 0x7f9628106060 2024-02-26T06:17:03.480 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.977+0000 7f9626000700 1 -- 172.21.15.6:0/1531510813 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96180042c0 con 0x7f9628106060 2024-02-26T06:17:03.482 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.977+0000 7f9626000700 1 -- 172.21.15.6:0/1531510813 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9618011a80 con 0x7f9628106060 2024-02-26T06:17:03.482 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 -- 172.21.15.6:0/1531510813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 msgr2=0x7f9628106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:03.482 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 --2- 172.21.15.6:0/1531510813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f9628106480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96180098a0 tx=0x7f9618009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:03.482 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 -- 172.21.15.6:0/1531510813 shutdown_connections 2024-02-26T06:17:03.482 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 --2- 172.21.15.6:0/1531510813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f9628106480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 -- 172.21.15.6:0/1531510813 >> 172.21.15.6:0/1531510813 conn(0x7f9628101c40 msgr2=0x7f9628104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 -- 172.21.15.6:0/1531510813 shutdown_connections 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.978+0000 7f962e400700 1 -- 172.21.15.6:0/1531510813 wait complete. 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.979+0000 7f962e400700 1 Processor -- start 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.979+0000 7f962e400700 1 -- start start 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.980+0000 7f962e400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f962819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:03.483 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.980+0000 7f962e400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962819dd10 con 0x7f9628106060 2024-02-26T06:17:03.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.980+0000 7f9627400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f962819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:03.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.980+0000 7f9627400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f962819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34100/0 (socket says 172.21.15.6:34100) 2024-02-26T06:17:03.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.980+0000 7f9627400700 1 -- 172.21.15.6:0/1601999323 learned_addr learned my addr 172.21.15.6:0/1601999323 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:03.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.980+0000 7f9627400700 1 -- 172.21.15.6:0/1601999323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618009550 con 0x7f9628106060 2024-02-26T06:17:03.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.981+0000 7f9627400700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f962819d7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9618009130 tx=0x7f9618011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:03.486 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.981+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9618003aa0 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.981+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9618003c00 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.981+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96180248a0 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.981+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962819df10 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.981+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962819eac0 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.982+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280632a0 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.982+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f961801ac10 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.983+0000 7f961fe00700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f960c038f30 0x7f960c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.983+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f961804d1e0 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.983+0000 7f9626a00700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f960c038f30 0x7f960c03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.984+0000 7f9626a00700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f960c038f30 0x7f960c03b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9614006fd0 tx=0x7f9614006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:01.993+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9618024a60 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.278+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f9628108050 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.763+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f961802c350 con 0x7f9628106060 2024-02-26T06:17:03.487 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.768+0000 7f961fe00700 1 -- 172.21.15.6:0/1601999323 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f961804f130 con 0x7f9628106060 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.769+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f960c038f30 msgr2=0x7f960c03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.769+0000 7f962e400700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f960c038f30 0x7f960c03b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9614006fd0 tx=0x7f9614006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.769+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 msgr2=0x7f962819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.769+0000 7f962e400700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f962819d7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9618009130 tx=0x7f9618011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.770+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 shutdown_connections 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.770+0000 7f962e400700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f960c038f30 0x7f960c03b3f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.770+0000 7f962e400700 1 --2- 172.21.15.6:0/1601999323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9628106060 0x7f962819d7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.770+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 >> 172.21.15.6:0/1601999323 conn(0x7f9628101c40 msgr2=0x7f96281944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.770+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 shutdown_connections 2024-02-26T06:17:03.488 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:02.770+0000 7f962e400700 1 -- 172.21.15.6:0/1601999323 wait complete. 2024-02-26T06:17:03.746 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:03.618+0000 7f99459d9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:17:04.040 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:03 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1601999323' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-26T06:17:04.040 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:03 smithi006 ceph-mon[31790]: mgrmap e9: x(active, since 26s) 2024-02-26T06:17:04.040 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:03.746+0000 7f99459d9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:17:04.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:04.577+0000 7f99459d9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:17:04.995 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:04 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1944549621' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T06:17:04.995 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:04.704+0000 7f99459d9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:17:04.996 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:04.889+0000 7f99459d9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:17:05.399 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:05.020+0000 7f99459d9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:17:05.399 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:05.134+0000 7f99459d9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:17:05.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:17:05.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-26T06:17:05.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.119+0000 7f0a05e00700 1 Processor -- start 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.119+0000 7f0a05e00700 1 -- start start 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.119+0000 7f0a05e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a00106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.120+0000 7f0a05e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00107050 con 0x7f0a00106660 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.120+0000 7f09fea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a00106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.120+0000 7f09fea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a00106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34136/0 (socket says 172.21.15.6:34136) 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.120+0000 7f09fea00700 1 -- 172.21.15.6:0/3578405216 learned_addr learned my addr 172.21.15.6:0/3578405216 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.121+0000 7f09fea00700 1 -- 172.21.15.6:0/3578405216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00107860 con 0x7f0a00106660 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.121+0000 7f09fea00700 1 --2- 172.21.15.6:0/3578405216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a00106a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09f00098a0 tx=0x7f09f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15863bede9787487 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:05.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.121+0000 7f09fe000700 1 -- 172.21.15.6:0/3578405216 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f0004160 con 0x7f0a00106660 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.121+0000 7f09fe000700 1 -- 172.21.15.6:0/3578405216 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09f00042c0 con 0x7f0a00106660 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.121+0000 7f09fe000700 1 -- 172.21.15.6:0/3578405216 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f0011a80 con 0x7f0a00106660 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 -- 172.21.15.6:0/3578405216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 msgr2=0x7f0a00106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 --2- 172.21.15.6:0/3578405216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a00106a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09f00098a0 tx=0x7f09f0009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 -- 172.21.15.6:0/3578405216 shutdown_connections 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 --2- 172.21.15.6:0/3578405216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a00106a80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 -- 172.21.15.6:0/3578405216 >> 172.21.15.6:0/3578405216 conn(0x7f0a00101c20 msgr2=0x7f0a00104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 -- 172.21.15.6:0/3578405216 shutdown_connections 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.122+0000 7f0a05e00700 1 -- 172.21.15.6:0/3578405216 wait complete. 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.123+0000 7f0a05e00700 1 Processor -- start 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.124+0000 7f0a05e00700 1 -- start start 2024-02-26T06:17:05.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.124+0000 7f0a05e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a000751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.124+0000 7f0a05e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0019c060 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.124+0000 7f09fea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a000751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.124+0000 7f09fea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a000751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34148/0 (socket says 172.21.15.6:34148) 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.124+0000 7f09fea00700 1 -- 172.21.15.6:0/1944549621 learned_addr learned my addr 172.21.15.6:0/1944549621 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f09fea00700 1 -- 172.21.15.6:0/1944549621 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f0009550 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f09fea00700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a000751d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09f0003f20 tx=0x7f09f0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f00038b0 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09f0003a10 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f001a5c0 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0019c260 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.125+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0019c700 con 0x7f0a00106660 2024-02-26T06:17:05.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.126+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f09f001a7c0 con 0x7f0a00106660 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.127+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0010aa90 con 0x7f0a00106660 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.127+0000 7f0a04a00700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 0x7f09e403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.127+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f09f004d240 con 0x7f0a00106660 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.127+0000 7f09f6000700 1 -- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 msgr2=0x7f09e403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.127+0000 7f09f6000700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 0x7f09e403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.132+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09f0024610 con 0x7f0a00106660 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.328+0000 7f09f6000700 1 -- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 msgr2=0x7f09e403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.328+0000 7f09f6000700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 0x7f09e403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.411+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0a0019cc70 con 0x7f0a00106660 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.412+0000 7f0a04a00700 1 -- 172.21.15.6:0/1944549621 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f09f0018350 con 0x7f0a00106660 2024-02-26T06:17:05.417 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.414+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 msgr2=0x7f09e403b400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.414+0000 7f0a05e00700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 0x7f09e403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.414+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 msgr2=0x7f0a000751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.414+0000 7f0a05e00700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a000751d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09f0003f20 tx=0x7f09f0011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.415+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 shutdown_connections 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.415+0000 7f0a05e00700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f09e4038f40 0x7f09e403b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.415+0000 7f0a05e00700 1 --2- 172.21.15.6:0/1944549621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a00106660 0x7f0a000751d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.415+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 >> 172.21.15.6:0/1944549621 conn(0x7f0a00101c20 msgr2=0x7f0a00102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.415+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 shutdown_connections 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:04.415+0000 7f0a05e00700 1 -- 172.21.15.6:0/1944549621 wait complete. 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-02-26T06:17:05.418 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 9... 2024-02-26T06:17:05.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:05.484+0000 7f99459d9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:17:05.672 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:05.630+0000 7f99459d9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:17:06.037 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:05.750+0000 7f99459d9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:17:06.037 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:05.881+0000 7f99459d9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:17:06.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:06.108+0000 7f99459d9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:17:07.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:07.433+0000 7f99459d9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:17:07.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:07.612+0000 7f99459d9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:17:08.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:08.041+0000 7f99459d9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:17:08.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:08.165+0000 7f99459d9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:17:08.685 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:08.432+0000 7f99459d9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:17:09.051 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:08.684+0000 7f99459d9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:17:09.401 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:09.075+0000 7f99459d9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:17:09.402 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:09.400+0000 7f99459d9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:17:10.080 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:09.715+0000 7f99459d9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:17:11.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:10.803+0000 7f99459d9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:17:11.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:11.528+0000 7f99459d9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:17:11.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:11.664+0000 7f99459d9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:17:12.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:17:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:17:11.955+0000 7f99459d9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:17:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: Active manager daemon x restarted 2024-02-26T06:17:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: Activating manager daemon x 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: osdmap e3: 0 total, 0 up, 0 in 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: mgrmap e10: x(active, starting, since 0.00908601s) 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:17:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:12 smithi006 ceph-mon[31790]: Manager daemon x is now available 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: [26/Feb/2024:06:17:12] ENGINE Bus STARTING 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: [26/Feb/2024:06:17:12] ENGINE Serving on https://172.21.15.6:7150 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: [26/Feb/2024:06:17:12] ENGINE Serving on http://172.21.15.6:8765 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: [26/Feb/2024:06:17:12] ENGINE Bus STARTED 2024-02-26T06:17:13.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:13 smithi006 ceph-mon[31790]: mgrmap e11: x(active, since 1.01044s) 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.036+0000 7f66f6e00700 1 Processor -- start 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.036+0000 7f66f6e00700 1 -- start start 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.037+0000 7f66f6e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:13.671 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.037+0000 7f66f6e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0108170 con 0x7f66f0107780 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.037+0000 7f66efe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.037+0000 7f66efe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f0107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34156/0 (socket says 172.21.15.6:34156) 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.037+0000 7f66efe00700 1 -- 172.21.15.6:0/1412649570 learned_addr learned my addr 172.21.15.6:0/1412649570 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.038+0000 7f66efe00700 1 -- 172.21.15.6:0/1412649570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0108980 con 0x7f66f0107780 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.038+0000 7f66efe00700 1 --2- 172.21.15.6:0/1412649570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f0107ba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009e80 tx=0x7f66e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfbcc6cd06b541 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.038+0000 7f66ef400700 1 -- 172.21.15.6:0/1412649570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66e000e070 con 0x7f66f0107780 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.038+0000 7f66ef400700 1 -- 172.21.15.6:0/1412649570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66e000b8c0 con 0x7f66f0107780 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.038+0000 7f66ef400700 1 -- 172.21.15.6:0/1412649570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66e00044a0 con 0x7f66f0107780 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.039+0000 7f66f6e00700 1 -- 172.21.15.6:0/1412649570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 msgr2=0x7f66f0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:13.672 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.039+0000 7f66f6e00700 1 --2- 172.21.15.6:0/1412649570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f0107ba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009e80 tx=0x7f66e000b330 comp rx=0 tx=0).stop 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.039+0000 7f66f6e00700 1 -- 172.21.15.6:0/1412649570 shutdown_connections 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.039+0000 7f66f6e00700 1 --2- 172.21.15.6:0/1412649570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f0107ba0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.039+0000 7f66f6e00700 1 -- 172.21.15.6:0/1412649570 >> 172.21.15.6:0/1412649570 conn(0x7f66f0102d80 msgr2=0x7f66f01051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.039+0000 7f66f6e00700 1 -- 172.21.15.6:0/1412649570 shutdown_connections 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.040+0000 7f66f6e00700 1 -- 172.21.15.6:0/1412649570 wait complete. 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66f6e00700 1 Processor -- start 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66f6e00700 1 -- start start 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66f6e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66f6e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f019ba80 con 0x7f66f0107780 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66efe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:13.673 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66efe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f019b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34164/0 (socket says 172.21.15.6:34164) 2024-02-26T06:17:13.674 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.041+0000 7f66efe00700 1 -- 172.21.15.6:0/2808095851 learned_addr learned my addr 172.21.15.6:0/2808095851 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:13.674 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.042+0000 7f66efe00700 1 -- 172.21.15.6:0/2808095851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0009b30 con 0x7f66f0107780 2024-02-26T06:17:13.674 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.043+0000 7f66efe00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f019b540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009fd0 tx=0x7f66e000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:13.674 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.043+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66e000e040 con 0x7f66f0107780 2024-02-26T06:17:13.674 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.043+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f019bc80 con 0x7f66f0107780 2024-02-26T06:17:13.674 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.043+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66e0003d60 con 0x7f66f0107780 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.043+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66e001c670 con 0x7f66f0107780 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.043+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f019c180 con 0x7f66f0107780 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.044+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f66e001c930 con 0x7f66f0107780 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.045+0000 7f66ed600700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.045+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 --> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66d403bb00 con 0x7f66d4038f30 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.045+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f66e004cf10 con 0x7f66f0107780 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.045+0000 7f66e7e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 msgr2=0x7f66d403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.045+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.246+0000 7f66e7e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 msgr2=0x7f66d403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:13.675 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.246+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.647+0000 7f66e7e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 msgr2=0x7f66d403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:06.647+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:07.448+0000 7f66e7e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 msgr2=0x7f66d403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:07.448+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:09.050+0000 7f66e7e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 msgr2=0x7f66d403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/4032437212 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:09.050+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:11.965+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7f66e001b4e0 con 0x7f66f0107780 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:11.965+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 msgr2=0x7f66d403b3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:11.965+0000 7f66ed600700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:13.676 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.965+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f66e001ada0 con 0x7f66f0107780 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.966+0000 7f66ed600700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66d403c9b0 0x7f66d403eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.966+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66d403bb00 con 0x7f66d403c9b0 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.967+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66d403c9b0 0x7f66d403eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.969+0000 7f66e7e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66d403c9b0 0x7f66d403eda0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f66d8003a10 tx=0x7f66d80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.971+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f66d403bb00 con 0x7f66d403c9b0 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.979+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f66f010bb80 con 0x7f66d403c9b0 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66ed600700 1 -- 172.21.15.6:0/2808095851 <== mgr.14152 v2:172.21.15.6:6800/324687139 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f66f010bb80 con 0x7f66d403c9b0 2024-02-26T06:17:13.677 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66d403c9b0 msgr2=0x7f66d403eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66d403c9b0 0x7f66d403eda0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f66d8003a10 tx=0x7f66d80092b0 comp rx=0 tx=0).stop 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 msgr2=0x7f66f019b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f019b540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009fd0 tx=0x7f66e000bd10 comp rx=0 tx=0).stop 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 shutdown_connections 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66d403c9b0 0x7f66d403eda0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66f0107780 0x7f66f019b540 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 --2- 172.21.15.6:0/2808095851 >> [v2:172.21.15.6:6800/4032437212,v1:172.21.15.6:6801/4032437212] conn(0x7f66d4038f30 0x7f66d403b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:13.678 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.980+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 >> 172.21.15.6:0/2808095851 conn(0x7f66f0102d80 msgr2=0x7f66f0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:13.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.981+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 shutdown_connections 2024-02-26T06:17:13.679 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:12.981+0000 7f66f6e00700 1 -- 172.21.15.6:0/2808095851 wait complete. 2024-02-26T06:17:13.679 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 9 is available 2024-02-26T06:17:13.679 INFO:teuthology.orchestra.run.smithi006.stdout:Generating a dashboard self-signed certificate... 2024-02-26T06:17:14.220 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:14 smithi006 ceph-mon[31790]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T06:17:14.220 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:14 smithi006 ceph-mon[31790]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T06:17:15.353 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:15 smithi006 ceph-mon[31790]: mgrmap e12: x(active, since 2s) 2024-02-26T06:17:15.353 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:15 smithi006 ceph-mon[31790]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:15.353 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:15.353 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:15.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.476+0000 7fa06d400700 1 Processor -- start 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.476+0000 7fa06d400700 1 -- start start 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.477+0000 7fa06d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa06810e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.477+0000 7fa06d400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06810e8e0 con 0x7fa06810df80 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.477+0000 7fa066800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa06810e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.477+0000 7fa066800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa06810e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47176/0 (socket says 172.21.15.6:47176) 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.477+0000 7fa066800700 1 -- 172.21.15.6:0/93539411 learned_addr learned my addr 172.21.15.6:0/93539411 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.478+0000 7fa066800700 1 -- 172.21.15.6:0/93539411 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06810ea20 con 0x7fa06810df80 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.478+0000 7fa066800700 1 --2- 172.21.15.6:0/93539411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa06810e3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098a0 tx=0x7fa058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d56f4a4fa6be31 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.478+0000 7fa065400700 1 -- 172.21.15.6:0/93539411 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058004160 con 0x7fa06810df80 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.478+0000 7fa065400700 1 -- 172.21.15.6:0/93539411 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0580042c0 con 0x7fa06810df80 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.478+0000 7fa065400700 1 -- 172.21.15.6:0/93539411 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058011a80 con 0x7fa06810df80 2024-02-26T06:17:15.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 -- 172.21.15.6:0/93539411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 msgr2=0x7fa06810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 --2- 172.21.15.6:0/93539411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa06810e3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098a0 tx=0x7fa058009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 -- 172.21.15.6:0/93539411 shutdown_connections 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 --2- 172.21.15.6:0/93539411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa06810e3a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 -- 172.21.15.6:0/93539411 >> 172.21.15.6:0/93539411 conn(0x7fa06807cde0 msgr2=0x7fa06807f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 -- 172.21.15.6:0/93539411 shutdown_connections 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.479+0000 7fa06d400700 1 -- 172.21.15.6:0/93539411 wait complete. 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.480+0000 7fa06d400700 1 Processor -- start 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.480+0000 7fa06d400700 1 -- start start 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa06d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa0681a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa06d400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681a2030 con 0x7fa06810df80 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa066800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa0681a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa066800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa0681a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47190/0 (socket says 172.21.15.6:47190) 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa066800700 1 -- 172.21.15.6:0/3268678424 learned_addr learned my addr 172.21.15.6:0/3268678424 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:15.755 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa066800700 1 -- 172.21.15.6:0/3268678424 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058009550 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.481+0000 7fa066800700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa0681a1af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa058009130 tx=0x7fa058011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.482+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058003aa0 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.482+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa058003c00 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.482+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0580248a0 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.482+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0681a2230 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.482+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0681a26d0 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.483+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fa05801ac10 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.483+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa068112230 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.483+0000 7fa05f400700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa04c038f90 0x7fa04c03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.484+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa05804d240 con 0x7fa06810df80 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.484+0000 7fa065e00700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa04c038f90 0x7fa04c03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.484+0000 7fa065e00700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa04c038f90 0x7fa04c03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa054006fd0 tx=0x7fa054006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:15.756 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.489+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa058018760 con 0x7fa06810df80 2024-02-26T06:17:15.757 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.724+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0681a4fb0 con 0x7fa04c038f90 2024-02-26T06:17:15.757 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.891+0000 7fa05f400700 1 -- 172.21.15.6:0/3268678424 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fa0681a4fb0 con 0x7fa04c038f90 2024-02-26T06:17:15.757 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa04c038f90 msgr2=0x7fa04c03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:15.757 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa04c038f90 0x7fa04c03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa054006fd0 tx=0x7fa054006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:15.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 msgr2=0x7fa0681a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:15.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa0681a1af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa058009130 tx=0x7fa058011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 shutdown_connections 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa04c038f90 0x7fa04c03b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 --2- 172.21.15.6:0/3268678424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa06810df80 0x7fa0681a1af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 >> 172.21.15.6:0/3268678424 conn(0x7fa06807cde0 msgr2=0x7fa06807d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.894+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 shutdown_connections 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:14.895+0000 7fa06d400700 1 -- 172.21.15.6:0/3268678424 wait complete. 2024-02-26T06:17:15.760 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial admin user... 2024-02-26T06:17:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:16 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:17 smithi006 ceph-mon[31790]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:17.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BYkN/dqtaPaDPWsvF2eKZuj.EzdpvwWTR9ynymDvHYoFYyhoD0sb6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708928236, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-26T06:17:17.799 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.421+0000 7f0ab4a00700 1 Processor -- start 2024-02-26T06:17:17.799 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.421+0000 7f0ab4a00700 1 -- start start 2024-02-26T06:17:17.799 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.422+0000 7f0ab4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:17.799 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.422+0000 7f0ab4a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab0108170 con 0x7f0ab0107780 2024-02-26T06:17:17.799 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.422+0000 7f0aade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:17.799 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.423+0000 7f0aade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab0107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47204/0 (socket says 172.21.15.6:47204) 2024-02-26T06:17:17.807 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.423+0000 7f0aade00700 1 -- 172.21.15.6:0/3592989392 learned_addr learned my addr 172.21.15.6:0/3592989392 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:17.807 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.423+0000 7f0aade00700 1 -- 172.21.15.6:0/3592989392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0108980 con 0x7f0ab0107780 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.423+0000 7f0aade00700 1 --2- 172.21.15.6:0/3592989392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab0107ba0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00098a0 tx=0x7f0aa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eec93d7c00ee11 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.423+0000 7f0aaca00700 1 -- 172.21.15.6:0/3592989392 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa0004160 con 0x7f0ab0107780 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.424+0000 7f0aaca00700 1 -- 172.21.15.6:0/3592989392 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0aa00042c0 con 0x7f0ab0107780 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.424+0000 7f0aaca00700 1 -- 172.21.15.6:0/3592989392 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa0011a80 con 0x7f0ab0107780 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.424+0000 7f0ab4a00700 1 -- 172.21.15.6:0/3592989392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 msgr2=0x7f0ab0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.424+0000 7f0ab4a00700 1 --2- 172.21.15.6:0/3592989392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab0107ba0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00098a0 tx=0x7f0aa0009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.425+0000 7f0ab4a00700 1 -- 172.21.15.6:0/3592989392 shutdown_connections 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.425+0000 7f0ab4a00700 1 --2- 172.21.15.6:0/3592989392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab0107ba0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.425+0000 7f0ab4a00700 1 -- 172.21.15.6:0/3592989392 >> 172.21.15.6:0/3592989392 conn(0x7f0ab0102d80 msgr2=0x7f0ab01051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.425+0000 7f0ab4a00700 1 -- 172.21.15.6:0/3592989392 shutdown_connections 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.425+0000 7f0ab4a00700 1 -- 172.21.15.6:0/3592989392 wait complete. 2024-02-26T06:17:17.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.426+0000 7f0ab4a00700 1 Processor -- start 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.426+0000 7f0ab4a00700 1 -- start start 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.426+0000 7f0ab4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab007ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.426+0000 7f0ab4a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab007cf80 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.426+0000 7f0aade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab007ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.427+0000 7f0aade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab007ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47212/0 (socket says 172.21.15.6:47212) 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.427+0000 7f0aade00700 1 -- 172.21.15.6:0/1972851524 learned_addr learned my addr 172.21.15.6:0/1972851524 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.427+0000 7f0aade00700 1 -- 172.21.15.6:0/1972851524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0009550 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.427+0000 7f0aade00700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab007ca40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.427+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa00039e0 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.428+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0aa0003b40 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.428+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab007b170 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.428+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0aa0024820 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.428+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab007b610 con 0x7f0ab0107780 2024-02-26T06:17:17.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.429+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f0aa001a430 con 0x7f0ab0107780 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.429+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab010bbb0 con 0x7f0ab0107780 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.430+0000 7f0aa6a00700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a94038f90 0x7f0a9403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.430+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0aa004d030 con 0x7f0ab0107780 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.430+0000 7f0aad400700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a94038f90 0x7f0a9403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.431+0000 7f0aad400700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a94038f90 0x7f0a9403b450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006fd0 tx=0x7f0a9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.436+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0aa001ab80 con 0x7f0ab0107780 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.687+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0ab00630c0 con 0x7f0a94038f90 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.956+0000 7f0aa6a00700 1 -- 172.21.15.6:0/1972851524 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f0ab00630c0 con 0x7f0a94038f90 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a94038f90 msgr2=0x7f0a9403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a94038f90 0x7f0a9403b450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006fd0 tx=0x7f0a9c006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 msgr2=0x7f0ab007ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab007ca40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa0011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 shutdown_connections 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a94038f90 0x7f0a9403b450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:17.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 --2- 172.21.15.6:0/1972851524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ab0107780 0x7f0ab007ca40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:17.811 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 >> 172.21.15.6:0/1972851524 conn(0x7f0ab0102d80 msgr2=0x7f0ab01038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:17.811 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.959+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 shutdown_connections 2024-02-26T06:17:17.811 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:16.960+0000 7f0ab4a00700 1 -- 172.21.15.6:0/1972851524 wait complete. 2024-02-26T06:17:17.811 INFO:teuthology.orchestra.run.smithi006.stdout:Fetching dashboard port number... 2024-02-26T06:17:18.686 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:19.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:19 smithi006 ceph-mon[31790]: mgrmap e13: x(active, since 6s) 2024-02-26T06:17:19.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:19 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3330715296' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-26T06:17:20.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 8443 2024-02-26T06:17:20.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.423+0000 7f8832200700 1 Processor -- start 2024-02-26T06:17:20.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.423+0000 7f8832200700 1 -- start start 2024-02-26T06:17:20.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.423+0000 7f8832200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:20.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.423+0000 7f8832200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c108150 con 0x7f882c107760 2024-02-26T06:17:20.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.424+0000 7f882b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:20.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.424+0000 7f882b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42386/0 (socket says 172.21.15.6:42386) 2024-02-26T06:17:20.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.424+0000 7f882b400700 1 -- 172.21.15.6:0/3353530668 learned_addr learned my addr 172.21.15.6:0/3353530668 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:20.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.424+0000 7f882b400700 1 -- 172.21.15.6:0/3353530668 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c108960 con 0x7f882c107760 2024-02-26T06:17:20.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.425+0000 7f882b400700 1 --2- 172.21.15.6:0/3353530668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c107b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f881c0098a0 tx=0x7f881c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a45f1f143079572 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:20.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.425+0000 7f882a000700 1 -- 172.21.15.6:0/3353530668 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f881c004160 con 0x7f882c107760 2024-02-26T06:17:20.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.425+0000 7f882a000700 1 -- 172.21.15.6:0/3353530668 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f881c0042c0 con 0x7f882c107760 2024-02-26T06:17:20.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.425+0000 7f882a000700 1 -- 172.21.15.6:0/3353530668 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f881c011a80 con 0x7f882c107760 2024-02-26T06:17:20.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 -- 172.21.15.6:0/3353530668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 msgr2=0x7f882c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:20.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 --2- 172.21.15.6:0/3353530668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c107b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f881c0098a0 tx=0x7f881c009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:20.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 -- 172.21.15.6:0/3353530668 shutdown_connections 2024-02-26T06:17:20.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 --2- 172.21.15.6:0/3353530668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c107b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:20.028 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 -- 172.21.15.6:0/3353530668 >> 172.21.15.6:0/3353530668 conn(0x7f882c102d00 msgr2=0x7f882c105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:20.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 -- 172.21.15.6:0/3353530668 shutdown_connections 2024-02-26T06:17:20.029 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.426+0000 7f8832200700 1 -- 172.21.15.6:0/3353530668 wait complete. 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.427+0000 7f8832200700 1 Processor -- start 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.427+0000 7f8832200700 1 -- start start 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.427+0000 7f8832200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.427+0000 7f8832200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19baa0 con 0x7f882c107760 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.428+0000 7f882b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.428+0000 7f882b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42392/0 (socket says 172.21.15.6:42392) 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.428+0000 7f882b400700 1 -- 172.21.15.6:0/3330715296 learned_addr learned my addr 172.21.15.6:0/3330715296 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.428+0000 7f882b400700 1 -- 172.21.15.6:0/3330715296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c009550 con 0x7f882c107760 2024-02-26T06:17:20.030 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.428+0000 7f882b400700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c19b560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f881c003f20 tx=0x7f881c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:20.031 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.429+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f881c0038b0 con 0x7f882c107760 2024-02-26T06:17:20.032 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.429+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f881c003a10 con 0x7f882c107760 2024-02-26T06:17:20.032 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.429+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f881c01a5c0 con 0x7f882c107760 2024-02-26T06:17:20.032 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.429+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c19bca0 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.429+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c19c140 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.430+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c10bb90 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.430+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f881c024430 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.431+0000 7f8823e00700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8810038ee0 0x7f881003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.431+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f881c04d340 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.431+0000 7f882aa00700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8810038ee0 0x7f881003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.432+0000 7f882aa00700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8810038ee0 0x7f881003b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8818006fd0 tx=0x7f8818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.437+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f881c018530 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.517+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f881c02cc50 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.685+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f882c19ea80 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.686+0000 7f8823e00700 1 -- 172.21.15.6:0/3330715296 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f881c018920 con 0x7f882c107760 2024-02-26T06:17:20.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8810038ee0 msgr2=0x7f881003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:20.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8810038ee0 0x7f881003b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8818006fd0 tx=0x7f8818006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 msgr2=0x7f882c19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c19b560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f881c003f20 tx=0x7f881c011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 shutdown_connections 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8810038ee0 0x7f881003b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 --2- 172.21.15.6:0/3330715296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c107760 0x7f882c19b560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.691+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 >> 172.21.15.6:0/3330715296 conn(0x7f882c102d00 msgr2=0x7f882c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.692+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 shutdown_connections 2024-02-26T06:17:20.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:18.692+0000 7f8832200700 1 -- 172.21.15.6:0/3330715296 wait complete. 2024-02-26T06:17:20.072 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T06:17:20.072 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-02-26T06:17:20.098 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T06:17:20.098 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-02-26T06:17:20.099 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-02-26T06:17:20.099 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-26T06:17:20.107 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph Dashboard is now available at: 2024-02-26T06:17:20.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:20.107 INFO:teuthology.orchestra.run.smithi006.stdout: URL: https://smithi006.front.sepia.ceph.com:8443/ 2024-02-26T06:17:20.107 INFO:teuthology.orchestra.run.smithi006.stdout: User: admin 2024-02-26T06:17:20.107 INFO:teuthology.orchestra.run.smithi006.stdout: Password: s372rmji88 2024-02-26T06:17:20.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:20.108 INFO:teuthology.orchestra.run.smithi006.stdout:Saving cluster configuration to /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config directory 2024-02-26T06:17:20.111 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling autotune for osd_memory_target 2024-02-26T06:17:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.069+0000 7fc41b000700 1 Processor -- start 2024-02-26T06:17:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.069+0000 7fc41b000700 1 -- start start 2024-02-26T06:17:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.069+0000 7fc41b000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc414107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.069+0000 7fc41b000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414108160 con 0x7fc414107770 2024-02-26T06:17:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.070+0000 7fc413e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc414107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.070+0000 7fc413e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc414107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42394/0 (socket says 172.21.15.6:42394) 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.070+0000 7fc413e00700 1 -- 172.21.15.6:0/3708270929 learned_addr learned my addr 172.21.15.6:0/3708270929 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.071+0000 7fc413e00700 1 -- 172.21.15.6:0/3708270929 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414108970 con 0x7fc414107770 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.071+0000 7fc413e00700 1 --2- 172.21.15.6:0/3708270929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc414107b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4040098a0 tx=0x7fc404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f436e77ab06cf6a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.072+0000 7fc412a00700 1 -- 172.21.15.6:0/3708270929 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc404004160 con 0x7fc414107770 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.072+0000 7fc412a00700 1 -- 172.21.15.6:0/3708270929 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4040042c0 con 0x7fc414107770 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.072+0000 7fc412a00700 1 -- 172.21.15.6:0/3708270929 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc404011a80 con 0x7fc414107770 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.072+0000 7fc41b000700 1 -- 172.21.15.6:0/3708270929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 msgr2=0x7fc414107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.072+0000 7fc41b000700 1 --2- 172.21.15.6:0/3708270929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc414107b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4040098a0 tx=0x7fc404009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.073+0000 7fc41b000700 1 -- 172.21.15.6:0/3708270929 shutdown_connections 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.073+0000 7fc41b000700 1 --2- 172.21.15.6:0/3708270929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc414107b90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.073+0000 7fc41b000700 1 -- 172.21.15.6:0/3708270929 >> 172.21.15.6:0/3708270929 conn(0x7fc414102d10 msgr2=0x7fc414105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.073+0000 7fc41b000700 1 -- 172.21.15.6:0/3708270929 shutdown_connections 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.073+0000 7fc41b000700 1 -- 172.21.15.6:0/3708270929 wait complete. 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.074+0000 7fc41b000700 1 Processor -- start 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.074+0000 7fc41b000700 1 -- start start 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.074+0000 7fc41b000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc41407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.074+0000 7fc41b000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41407e090 con 0x7fc414107770 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.075+0000 7fc413e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc41407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.075+0000 7fc413e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc41407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42404/0 (socket says 172.21.15.6:42404) 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.075+0000 7fc413e00700 1 -- 172.21.15.6:0/2300167378 learned_addr learned my addr 172.21.15.6:0/2300167378 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.075+0000 7fc413e00700 1 -- 172.21.15.6:0/2300167378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc404009550 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.075+0000 7fc413e00700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc41407db50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc404011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.076+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4040039e0 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.076+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc404003b40 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.076+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc404024820 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.076+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41407c280 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.076+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41407c720 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.077+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41410bba0 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.078+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fc40401ab80 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.078+0000 7fc410c00700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3f8039030 0x7fc3f803b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.078+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc40404d210 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.078+0000 7fc413400700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3f8039030 0x7fc3f803b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.079+0000 7fc413400700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3f8039030 0x7fc3f803b4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.084+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc40401a950 con 0x7fc414107770 2024-02-26T06:17:22.061 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.306+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc41410bdb0 con 0x7fc414107770 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.307+0000 7fc410c00700 1 -- 172.21.15.6:0/2300167378 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc40402c350 con 0x7fc414107770 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.309+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3f8039030 msgr2=0x7fc3f803b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.309+0000 7fc41b000700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3f8039030 0x7fc3f803b4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.309+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 msgr2=0x7fc41407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.309+0000 7fc41b000700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc41407db50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc404011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.310+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 shutdown_connections 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.310+0000 7fc41b000700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3f8039030 0x7fc3f803b4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.310+0000 7fc41b000700 1 --2- 172.21.15.6:0/2300167378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc414107770 0x7fc41407db50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.310+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 >> 172.21.15.6:0/2300167378 conn(0x7fc414102d10 msgr2=0x7fc414103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.310+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 shutdown_connections 2024-02-26T06:17:22.062 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:21.310+0000 7fc41b000700 1 -- 172.21.15.6:0/2300167378 wait complete. 2024-02-26T06:17:23.792 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.690+0000 7fac44600700 1 Processor -- start 2024-02-26T06:17:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.691+0000 7fac44600700 1 -- start start 2024-02-26T06:17:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.691+0000 7fac44600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c081bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.691+0000 7fac44600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c082180 con 0x7fac3c081790 2024-02-26T06:17:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.691+0000 7fac41c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c081bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.691+0000 7fac41c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c081bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42418/0 (socket says 172.21.15.6:42418) 2024-02-26T06:17:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.691+0000 7fac41c00700 1 -- 172.21.15.6:0/3237907152 learned_addr learned my addr 172.21.15.6:0/3237907152 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.692+0000 7fac41c00700 1 -- 172.21.15.6:0/3237907152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c10ef60 con 0x7fac3c081790 2024-02-26T06:17:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.692+0000 7fac41c00700 1 --2- 172.21.15.6:0/3237907152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c081bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac300098a0 tx=0x7fac30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65c62f5bd5be97b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.692+0000 7fac3be00700 1 -- 172.21.15.6:0/3237907152 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac30004160 con 0x7fac3c081790 2024-02-26T06:17:23.797 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac3be00700 1 -- 172.21.15.6:0/3237907152 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac300042c0 con 0x7fac3c081790 2024-02-26T06:17:23.797 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac3be00700 1 -- 172.21.15.6:0/3237907152 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac30011a80 con 0x7fac3c081790 2024-02-26T06:17:23.797 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 -- 172.21.15.6:0/3237907152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 msgr2=0x7fac3c081bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 --2- 172.21.15.6:0/3237907152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c081bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac300098a0 tx=0x7fac30009bb0 comp rx=0 tx=0).stop 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 -- 172.21.15.6:0/3237907152 shutdown_connections 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 --2- 172.21.15.6:0/3237907152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c081bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 -- 172.21.15.6:0/3237907152 >> 172.21.15.6:0/3237907152 conn(0x7fac3c07cd50 msgr2=0x7fac3c07f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 -- 172.21.15.6:0/3237907152 shutdown_connections 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.693+0000 7fac44600700 1 -- 172.21.15.6:0/3237907152 wait complete. 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.694+0000 7fac44600700 1 Processor -- start 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.695+0000 7fac44600700 1 -- start start 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.695+0000 7fac44600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c1a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.695+0000 7fac44600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c1a2060 con 0x7fac3c081790 2024-02-26T06:17:23.798 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.695+0000 7fac41c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c1a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.695+0000 7fac41c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c1a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42426/0 (socket says 172.21.15.6:42426) 2024-02-26T06:17:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.695+0000 7fac41c00700 1 -- 172.21.15.6:0/3248936355 learned_addr learned my addr 172.21.15.6:0/3248936355 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac41c00700 1 -- 172.21.15.6:0/3248936355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30009550 con 0x7fac3c081790 2024-02-26T06:17:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac41c00700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c1a1b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac30011740 tx=0x7fac30011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac300039e0 con 0x7fac3c081790 2024-02-26T06:17:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac30003b40 con 0x7fac3c081790 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c1a2260 con 0x7fac3c081790 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac30024740 con 0x7fac3c081790 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.696+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c1a2700 con 0x7fac3c081790 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.698+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fac3002d3f0 con 0x7fac3c081790 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.698+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3c112190 con 0x7fac3c081790 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.698+0000 7fac3a000700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac24039080 0x7fac2403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.698+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac3004d430 con 0x7fac3c081790 2024-02-26T06:17:23.802 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.698+0000 7fac41200700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac24039080 0x7fac2403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:23.802 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.699+0000 7fac41200700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac24039080 0x7fac2403b540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac28006fd0 tx=0x7fac28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:22.704+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac30010ca0 con 0x7fac3c081790 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.046+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fac3c1a4fe0 con 0x7fac3c081790 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.050+0000 7fac3a000700 1 -- 172.21.15.6:0/3248936355 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fac30010ed0 con 0x7fac3c081790 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac24039080 msgr2=0x7fac2403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac24039080 0x7fac2403b540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac28006fd0 tx=0x7fac28006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 msgr2=0x7fac3c1a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c1a1b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac30011740 tx=0x7fac30011b00 comp rx=0 tx=0).stop 2024-02-26T06:17:23.803 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 shutdown_connections 2024-02-26T06:17:23.804 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac24039080 0x7fac2403b540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:23.804 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 --2- 172.21.15.6:0/3248936355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac3c081790 0x7fac3c1a1b20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:23.804 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 >> 172.21.15.6:0/3248936355 conn(0x7fac3c07cd50 msgr2=0x7fac3c07d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:23.804 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 shutdown_connections 2024-02-26T06:17:23.804 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-02-26T06:17:23.053+0000 7fac44600700 1 -- 172.21.15.6:0/3248936355 wait complete. 2024-02-26T06:17:23.804 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:Or, if you are only running a single cluster on this host: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-26T06:17:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: ceph telemetry on 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:For more information see: 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:Bootstrap complete. 2024-02-26T06:17:23.917 INFO:tasks.cephadm:Fetching config... 2024-02-26T06:17:23.918 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:17:23.918 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-26T06:17:23.971 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-26T06:17:23.972 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:17:23.972 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-26T06:17:24.016 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-26T06:17:24.016 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:17:24.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-26T06:17:24.149 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-26T06:17:24.149 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:17:24.149 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-26T06:17:24.248 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-26T06:17:24.249 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T06:17:24.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:24 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3248936355' entity='client.admin' 2024-02-26T06:17:24.382 INFO:teuthology.orchestra.run.smithi006.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:17:24.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T06:17:24.491 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:17:24.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T06:17:24.552 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD20GcGHJPqbaODrsBxSYguX8zjK8Ex0YbTDt4mEOVkIgtnYSYQpuFhByTDrvhoNN9LzwG6DJf6izESecKcF6MXUHRRGQYnFrGFeZUgJs7MlF1bsMf/wrvRmko0PsI5PyKe1QuZhOdx7JiiE2GDNwkRbU32sGXp0TwUPfWDT18gmBTfNHuwaSAwT3SNjNGSVunii+bPClRpfY0Cvb0p6mKL+mj3gFw0g/pV1EtvFfTalG9MEOmk4jNxiYmfdJMQCrcz+C1+MEQ7X9nJVGWZ8Y499wxCgmQO4g2zkYKd+N60/EAtkxTIBcNeMw8/BDwtzP/qBB47Jv157Gf9lS5orIwpUo+UsbX2syHtcnyQ95UrRu8Q/nG+hVQob9y/D21E1NWjFw9DsLl7rNDApCrPKlWAq4G0NMHhDp6VXWnNFw5+cZJEJAwoPn9IMaazu2O24VZdxgIVpHSsMWLEB7cCbgl9/rGX8opHFC4UGkt3AIt6oYfAu2PRoFd/usOserG48s= ceph-5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:17:24.567 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-26T06:17:25.028 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 -- 172.21.15.6:0/3793772698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 msgr2=0x7f2428101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 --2- 172.21.15.6:0/3793772698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428101ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f241c009910 tx=0x7f241c009c20 comp rx=0 tx=0).stop 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 -- 172.21.15.6:0/3793772698 shutdown_connections 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 --2- 172.21.15.6:0/3793772698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428101ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 -- 172.21.15.6:0/3793772698 >> 172.21.15.6:0/3793772698 conn(0x7f24280fcc40 msgr2=0x7f24280ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 -- 172.21.15.6:0/3793772698 shutdown_connections 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.920+0000 7f242fe00700 1 -- 172.21.15.6:0/3793772698 wait complete. 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.922+0000 7f242fe00700 1 Processor -- start 2024-02-26T06:17:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.922+0000 7f242fe00700 1 -- start start 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.922+0000 7f242fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.922+0000 7f242fe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428075e70 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.923+0000 7f242d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.923+0000 7f242d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38922/0 (socket says 172.21.15.6:38922) 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.923+0000 7f242d400700 1 -- 172.21.15.6:0/3259393204 learned_addr learned my addr 172.21.15.6:0/3259393204 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.923+0000 7f242d400700 1 -- 172.21.15.6:0/3259393204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0095f0 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.923+0000 7f242d400700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428075930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f241c005850 tx=0x7f241c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.924+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f241c01d070 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.924+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f241c00be40 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.924+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f241c022ea0 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.924+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2428074060 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.924+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428074500 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.925+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f241c022670 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.925+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24280632a0 con 0x7f24281016a0 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.925+0000 7f2426000700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2410038fe0 0x7f241003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:28.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.926+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f241c04d0d0 con 0x7f24281016a0 2024-02-26T06:17:28.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.926+0000 7f242ca00700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2410038fe0 0x7f241003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:28.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.927+0000 7f242ca00700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2410038fe0 0x7f241003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2414006fd0 tx=0x7f2414006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:28.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:27.931+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f241c00b960 con 0x7f24281016a0 2024-02-26T06:17:28.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.162+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2428107530 con 0x7f24281016a0 2024-02-26T06:17:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.171+0000 7f2426000700 1 -- 172.21.15.6:0/3259393204 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f241c027070 con 0x7f24281016a0 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2410038fe0 msgr2=0x7f241003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2410038fe0 0x7f241003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2414006fd0 tx=0x7f2414006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 msgr2=0x7f2428075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428075930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f241c005850 tx=0x7f241c004ff0 comp rx=0 tx=0).stop 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 shutdown_connections 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2410038fe0 0x7f241003b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 --2- 172.21.15.6:0/3259393204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24281016a0 0x7f2428075930 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 >> 172.21.15.6:0/3259393204 conn(0x7f24280fcc40 msgr2=0x7f24280fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.174+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 shutdown_connections 2024-02-26T06:17:28.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:28.175+0000 7f242fe00700 1 -- 172.21.15.6:0/3259393204 wait complete. 2024-02-26T06:17:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:28.942 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-26T06:17:28.942 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-26T06:17:29.365 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:29.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:29 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3259393204' entity='client.admin' 2024-02-26T06:17:29.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:29.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.982+0000 7f365ac00700 1 -- 172.21.15.6:0/1328268737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 msgr2=0x7f3654101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.982+0000 7f365ac00700 1 --2- 172.21.15.6:0/1328268737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654101ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3644009910 tx=0x7f3644009c20 comp rx=0 tx=0).stop 2024-02-26T06:17:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.982+0000 7f365ac00700 1 -- 172.21.15.6:0/1328268737 shutdown_connections 2024-02-26T06:17:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.982+0000 7f365ac00700 1 --2- 172.21.15.6:0/1328268737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654101ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.982+0000 7f365ac00700 1 -- 172.21.15.6:0/1328268737 >> 172.21.15.6:0/1328268737 conn(0x7f36540fcc50 msgr2=0x7f36540ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.982+0000 7f365ac00700 1 -- 172.21.15.6:0/1328268737 shutdown_connections 2024-02-26T06:17:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.983+0000 7f365ac00700 1 -- 172.21.15.6:0/1328268737 wait complete. 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.983+0000 7f365ac00700 1 Processor -- start 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.984+0000 7f365ac00700 1 -- start start 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.984+0000 7f365ac00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.984+0000 7f365ac00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3654197110 con 0x7f36541016b0 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.984+0000 7f3653e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.984+0000 7f3653e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38930/0 (socket says 172.21.15.6:38930) 2024-02-26T06:17:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.984+0000 7f3653e00700 1 -- 172.21.15.6:0/867584128 learned_addr learned my addr 172.21.15.6:0/867584128 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.985+0000 7f3653e00700 1 -- 172.21.15.6:0/867584128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440095f0 con 0x7f36541016b0 2024-02-26T06:17:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.985+0000 7f3653e00700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654196bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3644004f40 tx=0x7f3644005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.985+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f364401d070 con 0x7f36541016b0 2024-02-26T06:17:31.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.985+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36440044a0 con 0x7f36541016b0 2024-02-26T06:17:31.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.986+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3654197310 con 0x7f36541016b0 2024-02-26T06:17:31.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.986+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3644003a40 con 0x7f36541016b0 2024-02-26T06:17:31.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.986+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36541977b0 con 0x7f36541016b0 2024-02-26T06:17:31.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.987+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f3644003cb0 con 0x7f36541016b0 2024-02-26T06:17:31.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.987+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3654190510 con 0x7f36541016b0 2024-02-26T06:17:31.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.987+0000 7f3650c00700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3638038fe0 0x7f363803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:31.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.987+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f364404cf10 con 0x7f36541016b0 2024-02-26T06:17:31.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.988+0000 7f3653400700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3638038fe0 0x7f363803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:31.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.988+0000 7f3653400700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3638038fe0 0x7f363803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3640006fd0 tx=0x7f3640006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:31.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:31.993+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3644004610 con 0x7f36541016b0 2024-02-26T06:17:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.219+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f36540620b0 con 0x7f3638038fe0 2024-02-26T06:17:32.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.226+0000 7f3650c00700 1 -- 172.21.15.6:0/867584128 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f36540620b0 con 0x7f3638038fe0 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3638038fe0 msgr2=0x7f363803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3638038fe0 0x7f363803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3640006fd0 tx=0x7f3640006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 msgr2=0x7f3654196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654196bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3644004f40 tx=0x7f3644005020 comp rx=0 tx=0).stop 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 shutdown_connections 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3638038fe0 0x7f363803b4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:32.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 --2- 172.21.15.6:0/867584128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36541016b0 0x7f3654196bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:32.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.229+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 >> 172.21.15.6:0/867584128 conn(0x7f36540fcc50 msgr2=0x7f36540fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:32.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.230+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 shutdown_connections 2024-02-26T06:17:32.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:32.230+0000 7f365ac00700 1 -- 172.21.15.6:0/867584128 wait complete. 2024-02-26T06:17:32.937 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi090 2024-02-26T06:17:32.937 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:17:32.937 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.conf 2024-02-26T06:17:32.955 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:17:32.955 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:17:33.012 INFO:tasks.cephadm:Adding host smithi090 to orchestrator... 2024-02-26T06:17:33.013 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch host add smithi090 2024-02-26T06:17:33.434 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:33.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:33 smithi006 ceph-mon[31790]: Updating smithi006:/etc/ceph/ceph.conf 2024-02-26T06:17:34.626 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:34 smithi006 ceph-mon[31790]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:17:34.627 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:34 smithi006 ceph-mon[31790]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:17:35.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 -- 172.21.15.6:0/3767866608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 msgr2=0x7f974c1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 --2- 172.21.15.6:0/3767866608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c1069e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f973c009910 tx=0x7f973c009c20 comp rx=0 tx=0).stop 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 -- 172.21.15.6:0/3767866608 shutdown_connections 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 --2- 172.21.15.6:0/3767866608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c1069e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 -- 172.21.15.6:0/3767866608 >> 172.21.15.6:0/3767866608 conn(0x7f974c0fc020 msgr2=0x7f974c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 -- 172.21.15.6:0/3767866608 shutdown_connections 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.421+0000 7f9750c00700 1 -- 172.21.15.6:0/3767866608 wait complete. 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.423+0000 7f9750c00700 1 Processor -- start 2024-02-26T06:17:35.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.423+0000 7f9750c00700 1 -- start start 2024-02-26T06:17:35.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.424+0000 7f9750c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:35.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.424+0000 7f9750c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c06a8b0 con 0x7f974c106600 2024-02-26T06:17:35.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.424+0000 7f9749e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:35.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.424+0000 7f9749e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38948/0 (socket says 172.21.15.6:38948) 2024-02-26T06:17:35.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.424+0000 7f9749e00700 1 -- 172.21.15.6:0/901047288 learned_addr learned my addr 172.21.15.6:0/901047288 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:35.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.425+0000 7f9749e00700 1 -- 172.21.15.6:0/901047288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0095f0 con 0x7f974c106600 2024-02-26T06:17:35.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.425+0000 7f9749e00700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c197ce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f973c006010 tx=0x7f973c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.425+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f973c01d070 con 0x7f974c106600 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.425+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f973c00bc30 con 0x7f974c106600 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.426+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f973c003930 con 0x7f974c106600 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.426+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c1982e0 con 0x7f974c106600 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.426+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c198760 con 0x7f974c106600 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.427+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f973c00b750 con 0x7f974c106600 2024-02-26T06:17:35.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.427+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c101c70 con 0x7f974c106600 2024-02-26T06:17:35.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.427+0000 7f9743400700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f972c038fe0 0x7f972c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:35.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.427+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f973c04ce10 con 0x7f974c106600 2024-02-26T06:17:35.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.428+0000 7f9741400700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f972c038fe0 0x7f972c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:35.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.428+0000 7f9741400700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f972c038fe0 0x7f972c03b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9734006fd0 tx=0x7f9734006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:35.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.433+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f973c003ae0 con 0x7f974c106600 2024-02-26T06:17:35.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:35.664+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}) v1 -- 0x7f974c198b80 con 0x7f972c038fe0 2024-02-26T06:17:35.665 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:35 smithi006 ceph-mon[31790]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:35.665 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:35 smithi006 ceph-mon[31790]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:17:35.665 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:35.665 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:35.665 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:36.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:36 smithi006 ceph-mon[31790]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:37.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:37 smithi006 ceph-mon[31790]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:37.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:37 smithi006 ceph-mon[31790]: Deploying cephadm binary to smithi090 2024-02-26T06:17:39.496 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:39 smithi006 ceph-mon[31790]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:40.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.630+0000 7f9743400700 1 -- 172.21.15.6:0/901047288 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f974c198b80 con 0x7f972c038fe0 2024-02-26T06:17:40.631 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi090' with addr '172.21.15.90' 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.633+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f972c038fe0 msgr2=0x7f972c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.633+0000 7f9750c00700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f972c038fe0 0x7f972c03b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9734006fd0 tx=0x7f9734006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.633+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 msgr2=0x7f974c197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.633+0000 7f9750c00700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c197ce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f973c006010 tx=0x7f973c004ff0 comp rx=0 tx=0).stop 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.634+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 shutdown_connections 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.634+0000 7f9750c00700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f972c038fe0 0x7f972c03b4a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:40.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.634+0000 7f9750c00700 1 --2- 172.21.15.6:0/901047288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f974c106600 0x7f974c197ce0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:40.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.634+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 >> 172.21.15.6:0/901047288 conn(0x7f974c0fc020 msgr2=0x7f974c0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:40.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.634+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 shutdown_connections 2024-02-26T06:17:40.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:40.634+0000 7f9750c00700 1 -- 172.21.15.6:0/901047288 wait complete. 2024-02-26T06:17:41.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T06:17:41.864 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:41 smithi006 ceph-mon[31790]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:41 smithi006 ceph-mon[31790]: Added host smithi090 2024-02-26T06:17:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:43.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:43.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:43 smithi006 ceph-mon[31790]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:43.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:44.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.002+0000 7f845ba00700 1 -- 172.21.15.6:0/865076742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 msgr2=0x7f8454107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:44.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.002+0000 7f845ba00700 1 --2- 172.21.15.6:0/865076742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454107a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f844800b0a0 tx=0x7f844800b3b0 comp rx=0 tx=0).stop 2024-02-26T06:17:44.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.002+0000 7f845ba00700 1 -- 172.21.15.6:0/865076742 shutdown_connections 2024-02-26T06:17:44.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.002+0000 7f845ba00700 1 --2- 172.21.15.6:0/865076742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454107a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:44.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.002+0000 7f845ba00700 1 -- 172.21.15.6:0/865076742 >> 172.21.15.6:0/865076742 conn(0x7f84540fd180 msgr2=0x7f84540ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:44.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.003+0000 7f845ba00700 1 -- 172.21.15.6:0/865076742 shutdown_connections 2024-02-26T06:17:44.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.003+0000 7f845ba00700 1 -- 172.21.15.6:0/865076742 wait complete. 2024-02-26T06:17:44.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.004+0000 7f845ba00700 1 Processor -- start 2024-02-26T06:17:44.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.004+0000 7f845ba00700 1 -- start start 2024-02-26T06:17:44.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.004+0000 7f845ba00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:44.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.004+0000 7f845ba00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84541996a0 con 0x7f8454107650 2024-02-26T06:17:44.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.005+0000 7f8459000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:44.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.005+0000 7f8459000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59974/0 (socket says 172.21.15.6:59974) 2024-02-26T06:17:44.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.005+0000 7f8459000700 1 -- 172.21.15.6:0/2273939890 learned_addr learned my addr 172.21.15.6:0/2273939890 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:44.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.005+0000 7f8459000700 1 -- 172.21.15.6:0/2273939890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8448009d00 con 0x7f8454107650 2024-02-26T06:17:44.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.005+0000 7f8459000700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454199160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84480062a0 tx=0x7f844800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:44.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.006+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f844800e040 con 0x7f8454107650 2024-02-26T06:17:44.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.006+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8448003c10 con 0x7f8454107650 2024-02-26T06:17:44.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.006+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f844801d600 con 0x7f8454107650 2024-02-26T06:17:44.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.006+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84541998a0 con 0x7f8454107650 2024-02-26T06:17:44.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.006+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8454199d20 con 0x7f8454107650 2024-02-26T06:17:44.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.007+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f8448019070 con 0x7f8454107650 2024-02-26T06:17:44.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.007+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84540632a0 con 0x7f8454107650 2024-02-26T06:17:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.008+0000 7f8451600700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f843c039020 0x7f843c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.008+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f844804d190 con 0x7f8454107650 2024-02-26T06:17:44.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.008+0000 7f8453e00700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f843c039020 0x7f843c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:44.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.009+0000 7f8453e00700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f843c039020 0x7f843c03b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8440006fd0 tx=0x7f8440006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:44.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.014+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8448003d80 con 0x7f8454107650 2024-02-26T06:17:44.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.246+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f845410bbd0 con 0x7f843c039020 2024-02-26T06:17:44.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.249+0000 7f8451600700 1 -- 172.21.15.6:0/2273939890 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f845410bbd0 con 0x7f843c039020 2024-02-26T06:17:44.248 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:44.248 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}] 2024-02-26T06:17:44.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f843c039020 msgr2=0x7f843c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:44.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f843c039020 0x7f843c03b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8440006fd0 tx=0x7f8440006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:44.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 msgr2=0x7f8454199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:44.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454199160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84480062a0 tx=0x7f844800bd40 comp rx=0 tx=0).stop 2024-02-26T06:17:44.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 shutdown_connections 2024-02-26T06:17:44.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f843c039020 0x7f843c03b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:44.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 --2- 172.21.15.6:0/2273939890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8454107650 0x7f8454199160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:44.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.251+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 >> 172.21.15.6:0/2273939890 conn(0x7f84540fd180 msgr2=0x7f84540ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:44.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.252+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 shutdown_connections 2024-02-26T06:17:44.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:44.252+0000 7f845ba00700 1 -- 172.21.15.6:0/2273939890 wait complete. 2024-02-26T06:17:44.966 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-02-26T06:17:44.967 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:17:44.967 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-02-26T06:17:44.985 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:17:44.985 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:17:45.041 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-02-26T06:17:45.042 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch host add smithi167 2024-02-26T06:17:45.512 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:45.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:45 smithi006 ceph-mon[31790]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:45.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:45 smithi006 ceph-mon[31790]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 -- 172.21.15.6:0/2553384086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 msgr2=0x7fd4cc1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 --2- 172.21.15.6:0/2553384086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc1009e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009910 tx=0x7fd4bc009c20 comp rx=0 tx=0).stop 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 -- 172.21.15.6:0/2553384086 shutdown_connections 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 --2- 172.21.15.6:0/2553384086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc1009e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 -- 172.21.15.6:0/2553384086 >> 172.21.15.6:0/2553384086 conn(0x7fd4cc075f00 msgr2=0x7fd4cc076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 -- 172.21.15.6:0/2553384086 shutdown_connections 2024-02-26T06:17:47.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.534+0000 7fd4cbe00700 1 -- 172.21.15.6:0/2553384086 wait complete. 2024-02-26T06:17:47.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.535+0000 7fd4cbe00700 1 Processor -- start 2024-02-26T06:17:47.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.536+0000 7fd4cbe00700 1 -- start start 2024-02-26T06:17:47.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.536+0000 7fd4cbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:47.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.536+0000 7fd4cbe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc199580 con 0x7fd4cc100600 2024-02-26T06:17:47.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.537+0000 7fd4caa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:47.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.537+0000 7fd4caa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47380/0 (socket says 172.21.15.6:47380) 2024-02-26T06:17:47.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.537+0000 7fd4caa00700 1 -- 172.21.15.6:0/3175032642 learned_addr learned my addr 172.21.15.6:0/3175032642 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:47.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.537+0000 7fd4caa00700 1 -- 172.21.15.6:0/3175032642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0095f0 con 0x7fd4cc100600 2024-02-26T06:17:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.537+0000 7fd4caa00700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc199040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006010 tx=0x7fd4bc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.538+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4bc01d070 con 0x7fd4cc100600 2024-02-26T06:17:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.538+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc00bc30 con 0x7fd4cc100600 2024-02-26T06:17:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.538+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4bc003930 con 0x7fd4cc100600 2024-02-26T06:17:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.538+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc199780 con 0x7fd4cc100600 2024-02-26T06:17:47.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.538+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc199ba0 con 0x7fd4cc100600 2024-02-26T06:17:47.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.539+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fd4bc003ae0 con 0x7fd4cc100600 2024-02-26T06:17:47.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.539+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc101ae0 con 0x7fd4cc100600 2024-02-26T06:17:47.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.540+0000 7fd4c3400700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd4b0038fe0 0x7fd4b003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:47.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.540+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4bc04cde0 con 0x7fd4cc100600 2024-02-26T06:17:47.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.540+0000 7fd4ca000700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd4b0038fe0 0x7fd4b003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:47.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.541+0000 7fd4ca000700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd4b0038fe0 0x7fd4b003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8006fd0 tx=0x7fd4b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:47.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.545+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4bc029540 con 0x7fd4cc100600 2024-02-26T06:17:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: Updating smithi090:/etc/ceph/ceph.conf 2024-02-26T06:17:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:47 smithi006 ceph-mon[31790]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:17:47.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:47.781+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4cc069da0 con 0x7fd4b0038fe0 2024-02-26T06:17:48.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:48 smithi006 ceph-mon[31790]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:17:48.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:48 smithi006 ceph-mon[31790]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:17:48.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:48 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:48.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:48 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:48.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:48 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:48.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:48 smithi006 ceph-mon[31790]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:17:49.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:49 smithi006 ceph-mon[31790]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:49.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:49 smithi006 ceph-mon[31790]: Deploying cephadm binary to smithi167 2024-02-26T06:17:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:51 smithi006 ceph-mon[31790]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:52.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.731+0000 7fd4c3400700 1 -- 172.21.15.6:0/3175032642 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd4cc069da0 con 0x7fd4b0038fe0 2024-02-26T06:17:52.731 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.734+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd4b0038fe0 msgr2=0x7fd4b003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.734+0000 7fd4cbe00700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd4b0038fe0 0x7fd4b003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8006fd0 tx=0x7fd4b8006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.734+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 msgr2=0x7fd4cc199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.734+0000 7fd4cbe00700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc199040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006010 tx=0x7fd4bc004ff0 comp rx=0 tx=0).stop 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.735+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 shutdown_connections 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.735+0000 7fd4cbe00700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd4b0038fe0 0x7fd4b003b4a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.735+0000 7fd4cbe00700 1 --2- 172.21.15.6:0/3175032642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4cc100600 0x7fd4cc199040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:52.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.735+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 >> 172.21.15.6:0/3175032642 conn(0x7fd4cc075f00 msgr2=0x7fd4cc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:52.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.735+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 shutdown_connections 2024-02-26T06:17:52.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:52.735+0000 7fd4cbe00700 1 -- 172.21.15.6:0/3175032642 wait complete. 2024-02-26T06:17:53.456 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T06:17:53.924 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:54.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:53 smithi006 ceph-mon[31790]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:54.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:54.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:53 smithi006 ceph-mon[31790]: Added host smithi167 2024-02-26T06:17:54.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:17:54.078 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:55.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:55.875 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:55 smithi006 ceph-mon[31790]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:55.875 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:55 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.048+0000 7f43c9200700 1 -- 172.21.15.6:0/1665339049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 msgr2=0x7f43c4102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.048+0000 7f43c9200700 1 --2- 172.21.15.6:0/1665339049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4102be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009910 tx=0x7f43b4009c20 comp rx=0 tx=0).stop 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.049+0000 7f43c9200700 1 -- 172.21.15.6:0/1665339049 shutdown_connections 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.049+0000 7f43c9200700 1 --2- 172.21.15.6:0/1665339049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4102be0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.049+0000 7f43c9200700 1 -- 172.21.15.6:0/1665339049 >> 172.21.15.6:0/1665339049 conn(0x7f43c40771b0 msgr2=0x7f43c40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.049+0000 7f43c9200700 1 -- 172.21.15.6:0/1665339049 shutdown_connections 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.049+0000 7f43c9200700 1 -- 172.21.15.6:0/1665339049 wait complete. 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.050+0000 7f43c9200700 1 Processor -- start 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.050+0000 7f43c9200700 1 -- start start 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.051+0000 7f43c9200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.051+0000 7f43c9200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4107590 con 0x7f43c4102800 2024-02-26T06:17:56.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.051+0000 7f43c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.051+0000 7f43c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4103680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47398/0 (socket says 172.21.15.6:47398) 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.051+0000 7f43c1e00700 1 -- 172.21.15.6:0/1848438045 learned_addr learned my addr 172.21.15.6:0/1848438045 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.052+0000 7f43c1e00700 1 -- 172.21.15.6:0/1848438045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40095f0 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.052+0000 7f43c1e00700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4103680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006010 tx=0x7f43b4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.052+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43b401d070 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.052+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b400bc30 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.052+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c4107790 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.052+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43b40039a0 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.053+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c4103d90 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.054+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c40632a0 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.054+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f43b4004390 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.054+0000 7f43baa00700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f43a8038fe0 0x7f43a803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.054+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f43b404cd70 con 0x7f43c4102800 2024-02-26T06:17:56.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.054+0000 7f43c1400700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f43a8038fe0 0x7f43a803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:56.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.055+0000 7f43c1400700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f43a8038fe0 0x7f43a803b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006fd0 tx=0x7f43b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:56.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.065+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43b4051070 con 0x7f43c4102800 2024-02-26T06:17:56.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.295+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43c410bbe0 con 0x7f43a8038fe0 2024-02-26T06:17:56.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.298+0000 7f43baa00700 1 -- 172.21.15.6:0/1848438045 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f43c410bbe0 con 0x7f43a8038fe0 2024-02-26T06:17:56.297 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:17:56.297 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-02-26T06:17:56.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.300+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f43a8038fe0 msgr2=0x7f43a803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:56.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.300+0000 7f43c9200700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f43a8038fe0 0x7f43a803b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006fd0 tx=0x7f43b0006e40 comp rx=0 tx=0).stop 2024-02-26T06:17:56.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 msgr2=0x7f43c4103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:56.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4103680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006010 tx=0x7f43b4004ff0 comp rx=0 tx=0).stop 2024-02-26T06:17:56.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 shutdown_connections 2024-02-26T06:17:56.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f43a8038fe0 0x7f43a803b4a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:56.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 --2- 172.21.15.6:0/1848438045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c4102800 0x7f43c4103680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:56.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 >> 172.21.15.6:0/1848438045 conn(0x7f43c40771b0 msgr2=0x7f43c41079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:56.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 shutdown_connections 2024-02-26T06:17:56.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:56.301+0000 7f43c9200700 1 -- 172.21.15.6:0/1848438045 wait complete. 2024-02-26T06:17:57.115 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-26T06:17:57.116 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd crush tunables default 2024-02-26T06:17:57.576 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:17:57.982 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:57 smithi006 ceph-mon[31790]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:57.982 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:57 smithi006 ceph-mon[31790]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:17:59.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.557+0000 7feaaec00700 1 -- 172.21.15.6:0/1106372254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 msgr2=0x7feaa8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:17:59.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.557+0000 7feaaec00700 1 --2- 172.21.15.6:0/1106372254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8107a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea98009910 tx=0x7fea98009c20 comp rx=0 tx=0).stop 2024-02-26T06:17:59.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.557+0000 7feaaec00700 1 -- 172.21.15.6:0/1106372254 shutdown_connections 2024-02-26T06:17:59.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.557+0000 7feaaec00700 1 --2- 172.21.15.6:0/1106372254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8107a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:17:59.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.558+0000 7feaaec00700 1 -- 172.21.15.6:0/1106372254 >> 172.21.15.6:0/1106372254 conn(0x7feaa80fd130 msgr2=0x7feaa80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:17:59.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.558+0000 7feaaec00700 1 -- 172.21.15.6:0/1106372254 shutdown_connections 2024-02-26T06:17:59.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.558+0000 7feaaec00700 1 -- 172.21.15.6:0/1106372254 wait complete. 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: Updating smithi167:/etc/ceph/ceph.conf 2024-02-26T06:17:59.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:17:59 smithi006 ceph-mon[31790]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:17:59.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.559+0000 7feaaec00700 1 Processor -- start 2024-02-26T06:17:59.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.560+0000 7feaaec00700 1 -- start start 2024-02-26T06:17:59.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.560+0000 7feaaec00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:59.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.560+0000 7feaaec00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa8199670 con 0x7feaa8107640 2024-02-26T06:17:59.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.560+0000 7feaa7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:59.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.560+0000 7feaa7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37016/0 (socket says 172.21.15.6:37016) 2024-02-26T06:17:59.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.560+0000 7feaa7e00700 1 -- 172.21.15.6:0/3641509117 learned_addr learned my addr 172.21.15.6:0/3641509117 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:17:59.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaa7e00700 1 -- 172.21.15.6:0/3641509117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980095f0 con 0x7feaa8107640 2024-02-26T06:17:59.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaa7e00700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8199130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea98006010 tx=0x7fea98004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:59.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea9801d070 con 0x7feaa8107640 2024-02-26T06:17:59.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea9800bbd0 con 0x7feaa8107640 2024-02-26T06:17:59.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa81998d0 con 0x7feaa8107640 2024-02-26T06:17:59.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea98003880 con 0x7feaa8107640 2024-02-26T06:17:59.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.561+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa8199d50 con 0x7feaa8107640 2024-02-26T06:17:59.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.562+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fea9800bd40 con 0x7feaa8107640 2024-02-26T06:17:59.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.562+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa8102d80 con 0x7feaa8107640 2024-02-26T06:17:59.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.563+0000 7feaa4c00700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fea8c038f80 0x7fea8c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:17:59.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.563+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea9804ce00 con 0x7feaa8107640 2024-02-26T06:17:59.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.563+0000 7feaa7400700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fea8c038f80 0x7fea8c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:17:59.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.564+0000 7feaa7400700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fea8c038f80 0x7fea8c03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea94006fd0 tx=0x7fea94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:17:59.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.568+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea9802ae20 con 0x7feaa8107640 2024-02-26T06:17:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:17:59.806+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7feaa810bb20 con 0x7feaa8107640 2024-02-26T06:18:00.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.706+0000 7feaa4c00700 1 -- 172.21.15.6:0/3641509117 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fea9802ac40 con 0x7feaa8107640 2024-02-26T06:18:00.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fea8c038f80 msgr2=0x7fea8c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:00.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fea8c038f80 0x7fea8c03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea94006fd0 tx=0x7fea94006e40 comp rx=0 tx=0).stop 2024-02-26T06:18:00.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 msgr2=0x7feaa8199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:00.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8199130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea98006010 tx=0x7fea98004ff0 comp rx=0 tx=0).stop 2024-02-26T06:18:00.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 shutdown_connections 2024-02-26T06:18:00.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fea8c038f80 0x7fea8c03b440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:00.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 --2- 172.21.15.6:0/3641509117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feaa8107640 0x7feaa8199130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:00.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.711+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 >> 172.21.15.6:0/3641509117 conn(0x7feaa80fd130 msgr2=0x7feaa8075910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:00.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.712+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 shutdown_connections 2024-02-26T06:18:00.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:00.712+0000 7feaaec00700 1 -- 172.21.15.6:0/3641509117 wait complete. 2024-02-26T06:18:00.713 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2024-02-26T06:18:01.079 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:00 smithi006 ceph-mon[31790]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:18:01.079 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:00 smithi006 ceph-mon[31790]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:18:01.079 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:01.079 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:01.079 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:01.080 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:00 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3641509117' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T06:18:01.486 INFO:tasks.cephadm:Adding mon.a on smithi006 2024-02-26T06:18:01.486 INFO:tasks.cephadm:Adding mon.b on smithi090 2024-02-26T06:18:01.486 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-02-26T06:18:01.486 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch apply mon '3;smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c' 2024-02-26T06:18:01.726 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:02.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:01 smithi006 ceph-mon[31790]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:02.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3641509117' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T06:18:02.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:01 smithi006 ceph-mon[31790]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:03.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 -- 172.21.15.167:0/2855846923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 msgr2=0x7f6c14107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:03.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 --2- 172.21.15.167:0/2855846923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14107ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009910 tx=0x7f6c04009c20 comp rx=0 tx=0).stop 2024-02-26T06:18:03.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 -- 172.21.15.167:0/2855846923 shutdown_connections 2024-02-26T06:18:03.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 --2- 172.21.15.167:0/2855846923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14107ab0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:03.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 -- 172.21.15.167:0/2855846923 >> 172.21.15.167:0/2855846923 conn(0x7f6c140fb7c0 msgr2=0x7f6c140fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:03.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 -- 172.21.15.167:0/2855846923 shutdown_connections 2024-02-26T06:18:03.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.197+0000 7f6c1b38c700 1 -- 172.21.15.167:0/2855846923 wait complete. 2024-02-26T06:18:03.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.198+0000 7f6c1b38c700 1 Processor -- start 2024-02-26T06:18:03.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.198+0000 7f6c1b38c700 1 -- start start 2024-02-26T06:18:03.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.198+0000 7f6c1b38c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:03.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.198+0000 7f6c1b38c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c14102b70 con 0x7f6c141076d0 2024-02-26T06:18:03.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.199+0000 7f6c19128700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:03.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.199+0000 7f6c19128700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.167:54212/0 (socket says 172.21.15.167:54212) 2024-02-26T06:18:03.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.199+0000 7f6c19128700 1 -- 172.21.15.167:0/651598282 learned_addr learned my addr 172.21.15.167:0/651598282 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:18:03.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.199+0000 7f6c19128700 1 -- 172.21.15.167:0/651598282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040095f0 con 0x7f6c141076d0 2024-02-26T06:18:03.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.200+0000 7f6c19128700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14100c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c04004f40 tx=0x7f6c04005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:03.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.200+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0401d070 con 0x7f6c141076d0 2024-02-26T06:18:03.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.200+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c141011c0 con 0x7f6c141076d0 2024-02-26T06:18:03.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.200+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0400bbd0 con 0x7f6c141076d0 2024-02-26T06:18:03.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.200+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c04003900 con 0x7f6c141076d0 2024-02-26T06:18:03.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.200+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c14101640 con 0x7f6c141076d0 2024-02-26T06:18:03.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.201+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c141021b0 con 0x7f6c141076d0 2024-02-26T06:18:03.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.201+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f6c0400bd40 con 0x7f6c141076d0 2024-02-26T06:18:03.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.202+0000 7f6c0a7fc700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6c00038f80 0x7f6c0003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:03.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.202+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c0404d320 con 0x7f6c141076d0 2024-02-26T06:18:03.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.202+0000 7f6c18927700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6c00038f80 0x7f6c0003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:03.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.203+0000 7f6c18927700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6c00038f80 0x7f6c0003b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:03.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.217+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c04021c60 con 0x7f6c141076d0 2024-02-26T06:18:03.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.422+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c141019a0 con 0x7f6c00038f80 2024-02-26T06:18:03.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.437+0000 7f6c0a7fc700 1 -- 172.21.15.167:0/651598282 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6c141019a0 con 0x7f6c00038f80 2024-02-26T06:18:03.437 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-02-26T06:18:03.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6c00038f80 msgr2=0x7f6c0003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:03.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6c00038f80 0x7f6c0003b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10006e40 comp rx=0 tx=0).stop 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 msgr2=0x7f6c14100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14100c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c04004f40 tx=0x7f6c04005020 comp rx=0 tx=0).stop 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 shutdown_connections 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6c00038f80 0x7f6c0003b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 --2- 172.21.15.167:0/651598282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c141076d0 0x7f6c14100c80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 >> 172.21.15.167:0/651598282 conn(0x7f6c140fb7c0 msgr2=0x7f6c140fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 shutdown_connections 2024-02-26T06:18:03.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:03.439+0000 7f6c1b38c700 1 -- 172.21.15.167:0/651598282 wait complete. 2024-02-26T06:18:04.102 DEBUG:teuthology.orchestra.run.smithi090:mon.b> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.b.service 2024-02-26T06:18:04.105 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.c.service 2024-02-26T06:18:04.107 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T06:18:04.107 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T06:18:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:05.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:04 smithi006 ceph-mon[31790]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:04 smithi006 ceph-mon[31790]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c;count:3 2024-02-26T06:18:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:04 smithi006 ceph-mon[31790]: Deploying daemon mon.c on smithi167 2024-02-26T06:18:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:05 smithi006 ceph-mon[31790]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 -- 172.21.15.167:0/1100296331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 msgr2=0x7f215c1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 --2- 172.21.15.167:0/1100296331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c1069e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f214c009910 tx=0x7f214c009c20 comp rx=0 tx=0).stop 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 -- 172.21.15.167:0/1100296331 shutdown_connections 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 --2- 172.21.15.167:0/1100296331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c1069e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 -- 172.21.15.167:0/1100296331 >> 172.21.15.167:0/1100296331 conn(0x7f215c0fc020 msgr2=0x7f215c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 -- 172.21.15.167:0/1100296331 shutdown_connections 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 -- 172.21.15.167:0/1100296331 wait complete. 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.835+0000 7f21616ee700 1 Processor -- start 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.836+0000 7f21616ee700 1 -- start start 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.836+0000 7f21616ee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:07.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.836+0000 7f21616ee700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c06a6f0 con 0x7f215c106600 2024-02-26T06:18:07.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.836+0000 7f215affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:07.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.836+0000 7f215affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.167:50342/0 (socket says 172.21.15.167:50342) 2024-02-26T06:18:07.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.836+0000 7f215affd700 1 -- 172.21.15.167:0/2542743665 learned_addr learned my addr 172.21.15.167:0/2542743665 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:18:07.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.837+0000 7f215affd700 1 -- 172.21.15.167:0/2542743665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0095f0 con 0x7f215c106600 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.837+0000 7f215affd700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c195ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f214c004f40 tx=0x7f214c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.838+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c01d070 con 0x7f215c106600 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.838+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c195ff0 con 0x7f215c106600 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.838+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c1964f0 con 0x7f215c106600 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.838+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f214c00bc30 con 0x7f215c106600 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.838+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c003970 con 0x7f215c106600 2024-02-26T06:18:07.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.839+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2148005330 con 0x7f215c106600 2024-02-26T06:18:07.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.839+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f214c003ad0 con 0x7f215c106600 2024-02-26T06:18:07.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.839+0000 7f2143fff700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2144038fe0 0x7f214403b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:07.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.839+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f214c04ccd0 con 0x7f215c106600 2024-02-26T06:18:07.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.840+0000 7f215a7fc700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2144038fe0 0x7f214403b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:07.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.841+0000 7f215a7fc700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2144038fe0 0x7f214403b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2150006fd0 tx=0x7f2150006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:07.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:07.855+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f214c00b6f0 con 0x7f215c106600 2024-02-26T06:18:07.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:07 smithi167 systemd[1]: Starting Ceph mon.c for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:18:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.152+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f21480058b0 con 0x7f215c106600 2024-02-26T06:18:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.153+0000 7f2143fff700 1 -- 172.21.15.167:0/2542743665 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+741 (secure 0 0 0) 0x7f214c030070 con 0x7f215c106600 2024-02-26T06:18:08.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:18:08.153 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","modified":"2024-02-26T06:15:58.485836Z","created":"2024-02-26T06:15:58.485836Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2144038fe0 msgr2=0x7f214403b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2144038fe0 0x7f214403b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2150006fd0 tx=0x7f2150006e40 comp rx=0 tx=0).stop 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 msgr2=0x7f215c195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c195ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f214c004f40 tx=0x7f214c005020 comp rx=0 tx=0).stop 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 shutdown_connections 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2144038fe0 0x7f214403b4a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 --2- 172.21.15.167:0/2542743665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f215c106600 0x7f215c195ab0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 >> 172.21.15.167:0/2542743665 conn(0x7f215c0fc020 msgr2=0x7f215c0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 shutdown_connections 2024-02-26T06:18:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:08.155+0000 7f21616ee700 1 -- 172.21.15.167:0/2542743665 wait complete. 2024-02-26T06:18:08.156 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-02-26T06:18:08.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:07 smithi006 ceph-mon[31790]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:08.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 podman[36195]: 2024-02-26 06:18:07.989394336 +0000 UTC m=+0.018030779 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:18:08.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 podman[36195]: 2024-02-26 06:18:08.091292047 +0000 UTC m=+0.119928483 container create fd386e6b9e0d77facb10a6d47cea64626ba9df382a144cfa216c53c7afc3fbfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD) 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 podman[36195]: 2024-02-26 06:18:08.830597078 +0000 UTC m=+0.859233516 container init fd386e6b9e0d77facb10a6d47cea64626ba9df382a144cfa216c53c7afc3fbfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 podman[36195]: 2024-02-26 06:18:08.834182407 +0000 UTC m=+0.862818830 container start fd386e6b9e0d77facb10a6d47cea64626ba9df382a144cfa216c53c7afc3fbfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev), process ceph-mon, pid 2 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pidfile_write: ignore empty --pid-file 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: load: jerasure load: lrc 2024-02-26T06:18:08.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: RocksDB version: 7.9.2 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Git sha 0 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Compile date 2024-02-16 06:18:30 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: DB SUMMARY 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: DB Session ID: 9Q4SWXSWKIGSRDYUKBXC 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: CURRENT file: CURRENT 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.error_if_exists: 0 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.create_if_missing: 0 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.paranoid_checks: 1 2024-02-26T06:18:08.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.env: 0x5603005a9ae0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.info_log: 0x5603022e98c0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.statistics: (nil) 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.use_fsync: 0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_log_file_size: 0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.allow_fallocate: 1 2024-02-26T06:18:08.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.use_direct_reads: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.db_log_dir: 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.wal_dir: 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T06:18:08.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.write_buffer_manager: 0x5603022bd400 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.unordered_write: 0 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.row_cache: None 2024-02-26T06:18:08.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.wal_filter: None 2024-02-26T06:18:09.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T06:18:09.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T06:18:09.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.two_write_queues: 0 2024-02-26T06:18:09.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.wal_compression: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.atomic_flush: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.log_readahead_size: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T06:18:09.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_background_jobs: 2 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_background_compactions: -1 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_subcompactions: 1 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_open_files: -1 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T06:18:09.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_background_flushes: -1 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Compression algorithms supported: 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kZSTD supported: 0 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kXpressCompression supported: 0 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kZlibCompression supported: 1 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kSnappyCompression supported: 1 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kLZ4Compression supported: 1 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: kBZip2Compression supported: 0 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-26T06:18:09.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.merge_operator: 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_filter: None 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5603022e99c0) 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks: 1 2024-02-26T06:18:09.005 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_top_level_index_and_filter: 1 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_type: 0 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_index_type: 0 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_shortening: 1 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: checksum: 4 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: no_block_cache: 0 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache: 0x5603022ed090 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_name: BinnedLRUCache 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_options: 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: capacity : 536870912 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_shard_bits : 4 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: strict_capacity_limit : 0 2024-02-26T06:18:09.006 INFO:journalctl@ceph.mon.c.smithi167.stdout: high_pri_pool_ratio: 0.000 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_compressed: (nil) 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: persistent_cache: (nil) 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size: 4096 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size_deviation: 10 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_restart_interval: 16 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_block_restart_interval: 1 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: metadata_block_size: 4096 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: partition_filters: 0 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: use_delta_encoding: 1 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: filter_policy: bloomfilter 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: whole_key_filtering: 1 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: verify_compression: 0 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: read_amp_bytes_per_bit: 0 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: format_version: 5 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_index_compression: 1 2024-02-26T06:18:09.007 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_align: 0 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout: max_auto_readahead_size: 262144 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout: prepopulate_block_cache: 0 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout: initial_auto_readahead_size: 8192 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression: NoCompression 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.num_levels: 7 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T06:18:09.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T06:18:09.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T06:18:09.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T06:18:09.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.table_properties_collectors: 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.inplace_update_support: 0 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.bloom_locality: 0 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.max_successive_merges: 0 2024-02-26T06:18:09.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.ttl: 2592000 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.enable_blob_files: false 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.min_blob_size: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T06:18:09.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 269b403a-01b4-48ad-a9d0-5f0c678dd82a 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928288864219, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928288864653, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708928288, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "269b403a-01b4-48ad-a9d0-5f0c678dd82a", "db_session_id": "9Q4SWXSWKIGSRDYUKBXC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708928288864732, "job": 1, "event": "recovery_finished"} 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5603023de000 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: DB pointer 0x5603023c8000 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T06:18:09.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** DB Stats ** 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T06:18:09.015 INFO:journalctl@ceph.mon.c.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T06:18:09.016 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache BinnedLRUCache@0x5603022ed090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: using public_addr v2:172.21.15.167:0/0 -> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: starting mon.c rank -1 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(???) e0 preinit fsid 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).mds e1 new map 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-26T06:18:09.017 INFO:journalctl@ceph.mon.c.smithi167.stdout: e1 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout: legacy client fscid: -1 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout: No filesystems configured 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:18:09.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Deploying cephadm binary to smithi090 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Added host smithi090 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:18:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi090:/etc/ceph/ceph.conf 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Deploying cephadm binary to smithi167 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Added host smithi167 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi167:/etc/ceph/ceph.conf 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3641509117' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3641509117' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T06:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c;count:3 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: Deploying daemon mon.c on smithi167 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/2542743665' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T06:18:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 ceph-mon[36229]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-26T06:18:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 bash[36195]: fd386e6b9e0d77facb10a6d47cea64626ba9df382a144cfa216c53c7afc3fbfd 2024-02-26T06:18:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:08 smithi167 systemd[1]: Started Ceph mon.c for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:18:09.073 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:08 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/2542743665' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T06:18:09.747 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T06:18:09.747 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T06:18:09.990 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: mon.a calling monitor election 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: mon.c calling monitor election 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: fsmap 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: mgrmap e13: x(active, since 61s) 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: overall HEALTH_OK 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:14.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: mon.a calling monitor election 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:14.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: mon.c calling monitor election 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: fsmap 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: mgrmap e13: x(active, since 61s) 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: overall HEALTH_OK 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:14.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.457+0000 7fe674842700 1 -- 172.21.15.167:0/3795676867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6580057d0 msgr2=0x7fe658007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:14.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.457+0000 7fe674842700 1 --2- 172.21.15.167:0/3795676867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6580057d0 0x7fe658007c60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe670100570 tx=0x7fe66000feb0 comp rx=0 tx=0).stop 2024-02-26T06:18:14.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.458+0000 7fe674842700 1 -- 172.21.15.167:0/3795676867 shutdown_connections 2024-02-26T06:18:14.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.458+0000 7fe674842700 1 --2- 172.21.15.167:0/3795676867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6580057d0 0x7fe658007c60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:14.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.458+0000 7fe674842700 1 -- 172.21.15.167:0/3795676867 >> 172.21.15.167:0/3795676867 conn(0x7fe6700fc110 msgr2=0x7fe6700fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:14.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.458+0000 7fe674842700 1 -- 172.21.15.167:0/3795676867 shutdown_connections 2024-02-26T06:18:14.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.458+0000 7fe674842700 1 -- 172.21.15.167:0/3795676867 wait complete. 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe674842700 1 Processor -- start 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe674842700 1 -- start start 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe674842700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe674842700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe670195390 0x7fe670199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe674842700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670195920 con 0x7fe670195390 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe674842700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670195a90 con 0x7fe6580057d0 2024-02-26T06:18:14.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.459+0000 7fe66e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55308/0 (socket says 172.21.15.167:55308) 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66e59c700 1 -- 172.21.15.167:0/1607460293 learned_addr learned my addr 172.21.15.167:0/1607460293 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66e59c700 1 -- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 msgr2=0x7fe670194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66e59c700 1 -- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 msgr2=0x7fe670194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66e59c700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66e59c700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:18:14.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.460+0000 7fe66dd9b700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe670195390 0x7fe670199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.461+0000 7fe66dd9b700 1 -- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 msgr2=0x7fe670194e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.461+0000 7fe66dd9b700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.461+0000 7fe66dd9b700 1 -- 172.21.15.167:0/1607460293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66000f900 con 0x7fe670195390 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.461+0000 7fe66dd9b700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe670195390 0x7fe670199720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b4d0 tx=0x7fe66400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.462+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe664011630 con 0x7fe670195390 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.462+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe670199cc0 con 0x7fe670195390 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.462+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe664011c70 con 0x7fe670195390 2024-02-26T06:18:14.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.462+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe664010de0 con 0x7fe670195390 2024-02-26T06:18:14.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.462+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67019a210 con 0x7fe670195390 2024-02-26T06:18:14.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.463+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67018edd0 con 0x7fe670195390 2024-02-26T06:18:14.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.463+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fe6640104a0 con 0x7fe670195390 2024-02-26T06:18:14.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.463+0000 7fe65f7fe700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe650038d20 0x7fe65003b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:14.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.464+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe66404b810 con 0x7fe670195390 2024-02-26T06:18:14.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.464+0000 7fe66e59c700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe650038d20 0x7fe65003b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:14.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.465+0000 7fe66e59c700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe650038d20 0x7fe65003b1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe660001eb0 tx=0x7fe660005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:14.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.480+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe664011de0 con 0x7fe670195390 2024-02-26T06:18:14.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.754+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe6701961b0 con 0x7fe670195390 2024-02-26T06:18:14.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.755+0000 7fe65f7fe700 1 -- 172.21.15.167:0/1607460293 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7fe66403b020 con 0x7fe670195390 2024-02-26T06:18:14.755 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:18:14.755 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","modified":"2024-02-26T06:18:08.896180Z","created":"2024-02-26T06:15:58.485836Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe650038d20 msgr2=0x7fe65003b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe650038d20 0x7fe65003b1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe660001eb0 tx=0x7fe660005960 comp rx=0 tx=0).stop 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe670195390 msgr2=0x7fe670199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe670195390 0x7fe670199720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b4d0 tx=0x7fe66400b890 comp rx=0 tx=0).stop 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 shutdown_connections 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe650038d20 0x7fe65003b1e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6580057d0 0x7fe670194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:14.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 --2- 172.21.15.167:0/1607460293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe670195390 0x7fe670199720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:14.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 >> 172.21.15.167:0/1607460293 conn(0x7fe6700fc110 msgr2=0x7fe6700fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:14.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 shutdown_connections 2024-02-26T06:18:14.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:14.757+0000 7fe674842700 1 -- 172.21.15.167:0/1607460293 wait complete. 2024-02-26T06:18:14.759 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: Deploying daemon mon.b on smithi090 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/1607460293' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T06:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:15.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: Deploying daemon mon.b on smithi090 2024-02-26T06:18:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/1607460293' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T06:18:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:16.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:18:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:18:15.896+0000 7f98fac00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-26T06:18:16.344 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T06:18:16.345 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T06:18:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:18:17.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:17 smithi167 ceph-mon[36229]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:17 smithi006 ceph-mon[31790]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:18.004 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:17 smithi090 ceph-mon[35681]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-26T06:18:18.004 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:18 smithi090 bash[35661]: 09d413e3f1140c433e9782e3a30cbf9ee1de2625293616dfe4a4656802c01b16 2024-02-26T06:18:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:18 smithi090 systemd[1]: Started Ceph mon.b for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:18:23.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:23.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: mon.c calling monitor election 2024-02-26T06:18:23.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:18:23.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: mon.a calling monitor election 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: fsmap 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: mgrmap e13: x(active, since 71s) 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T06:18:23.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T06:18:23.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T06:18:23.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: mon.b (rank 2) addr [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] is down (out of quorum) 2024-02-26T06:18:23.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:23.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:23.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: mon.c calling monitor election 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: mon.a calling monitor election 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:23.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: fsmap 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: mgrmap e13: x(active, since 71s) 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: mon.b (rank 2) addr [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] is down (out of quorum) 2024-02-26T06:18:23.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:23.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:24.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 -- 172.21.15.167:0/3645118513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc088008450 msgr2=0x7fc08800a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:24.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 --2- 172.21.15.167:0/3645118513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc088008450 0x7fc08800a8c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0100f00 tx=0x7fc090010ea0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 -- 172.21.15.167:0/3645118513 shutdown_connections 2024-02-26T06:18:24.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 --2- 172.21.15.167:0/3645118513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc088008450 0x7fc08800a8c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 -- 172.21.15.167:0/3645118513 >> 172.21.15.167:0/3645118513 conn(0x7fc0a00fbd80 msgr2=0x7fc0a00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:24.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 -- 172.21.15.167:0/3645118513 shutdown_connections 2024-02-26T06:18:24.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.003+0000 7fc0a6861700 1 -- 172.21.15.167:0/3645118513 wait complete. 2024-02-26T06:18:24.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 Processor -- start 2024-02-26T06:18:24.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 -- start start 2024-02-26T06:18:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc088008450 0x7fc0a0195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a01955d0 0x7fc0a0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 0x7fc0a019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0195cc0 con 0x7fc0a01955d0 2024-02-26T06:18:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01a1c10 con 0x7fc088008450 2024-02-26T06:18:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.004+0000 7fc0a6861700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01a1d50 con 0x7fc0a0199ea0 2024-02-26T06:18:24.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 0x7fc0a019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:24.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 0x7fc0a019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33536/0 (socket says 172.21.15.167:33536) 2024-02-26T06:18:24.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 -- 172.21.15.167:0/354781827 learned_addr learned my addr 172.21.15.167:0/354781827 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:18:24.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 -- 172.21.15.167:0/354781827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc088008450 msgr2=0x7fc0a0195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:24.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc088008450 0x7fc0a0195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 -- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a01955d0 msgr2=0x7fc0a0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a01955d0 0x7fc0a0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 -- 172.21.15.167:0/354781827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0900108f0 con 0x7fc0a0199ea0 2024-02-26T06:18:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a4dfe700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 0x7fc0a019a350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc09800ba50 tx=0x7fc09800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800cab0 con 0x7fc0a0199ea0 2024-02-26T06:18:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc09800cc10 con 0x7fc0a0199ea0 2024-02-26T06:18:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a01a2030 con 0x7fc0a0199ea0 2024-02-26T06:18:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0980075d0 con 0x7fc0a0199ea0 2024-02-26T06:18:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.005+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a01a2580 con 0x7fc0a0199ea0 2024-02-26T06:18:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.006+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fc098007770 con 0x7fc0a0199ea0 2024-02-26T06:18:24.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.006+0000 7fc09d7fa700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc08c039410 0x7fc08c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:24.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.006+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc09800cd80 con 0x7fc0a0199ea0 2024-02-26T06:18:24.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.006+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a018ed20 con 0x7fc0a0199ea0 2024-02-26T06:18:24.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.007+0000 7fc09ffff700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc08c039410 0x7fc08c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:24.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.007+0000 7fc09ffff700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc08c039410 0x7fc08c03b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc090005b40 tx=0x7fc090005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:24.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.011+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc09801b8b0 con 0x7fc0a0199ea0 2024-02-26T06:18:24.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.291+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc0a0107370 con 0x7fc0a0199ea0 2024-02-26T06:18:24.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:24.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:24.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:24.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:24.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.291+0000 7fc09d7fa700 1 -- 172.21.15.167:0/354781827 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1275 (secure 0 0 0) 0x7fc09801b2c0 con 0x7fc0a0199ea0 2024-02-26T06:18:24.292 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-02-26T06:18:24.292 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","modified":"2024-02-26T06:18:17.985514Z","created":"2024-02-26T06:15:58.485836Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc08c039410 msgr2=0x7fc08c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc08c039410 0x7fc08c03b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc090005b40 tx=0x7fc090005ab0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 msgr2=0x7fc0a019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 0x7fc0a019a350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc09800ba50 tx=0x7fc09800be10 comp rx=0 tx=0).stop 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 shutdown_connections 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc08c039410 0x7fc08c03b8d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0a0199ea0 0x7fc0a019a350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc088008450 0x7fc0a0195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 --2- 172.21.15.167:0/354781827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a01955d0 0x7fc0a0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:24.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 >> 172.21.15.167:0/354781827 conn(0x7fc0a00fbd80 msgr2=0x7fc0a00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:24.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 shutdown_connections 2024-02-26T06:18:24.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:24.294+0000 7fc0a6861700 1 -- 172.21.15.167:0/354781827 wait complete. 2024-02-26T06:18:24.295 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-02-26T06:18:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:24.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:24.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.331 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-26T06:18:25.331 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph config generate-minimal-conf 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.c calling monitor election 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.a calling monitor election 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:18:25.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: fsmap 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mgrmap e13: x(active, since 71s) 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T06:18:25.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.b (rank 2) addr [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] is down (out of quorum) 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:25.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:25.752 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:18:26.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: mon.b calling monitor election 2024-02-26T06:18:26.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: mon.b calling monitor election 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: mon.c calling monitor election 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: mon.a calling monitor election 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: fsmap 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: mgrmap e13: x(active, since 73s) 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: Cluster is now healthy 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: overall HEALTH_OK 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: mon.b calling monitor election 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: mon.b calling monitor election 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: mon.c calling monitor election 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: mon.a calling monitor election 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: fsmap 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: mgrmap e13: x(active, since 73s) 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: Cluster is now healthy 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: overall HEALTH_OK 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.b calling monitor election 2024-02-26T06:18:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.b calling monitor election 2024-02-26T06:18:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.c calling monitor election 2024-02-26T06:18:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.a calling monitor election 2024-02-26T06:18:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T06:18:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: fsmap 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: mgrmap e13: x(active, since 73s) 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: Cluster is now healthy 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: overall HEALTH_OK 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:26.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:18:26.986+0000 7f98fac00700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:27.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:27.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:26 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.183 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.184 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.184 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.184 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.184 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:27.184 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:27.184 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:26 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:27.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:27.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:26 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:27.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 -- 172.21.15.6:0/2760314696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 msgr2=0x7f7868107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:27.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 --2- 172.21.15.6:0/2760314696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868107960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7858009910 tx=0x7f7858009c20 comp rx=0 tx=0).stop 2024-02-26T06:18:27.989 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:27 smithi006 ceph-mon[31790]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:27.989 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:27 smithi006 ceph-mon[31790]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T06:18:27.989 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:27 smithi006 ceph-mon[31790]: Reconfiguring daemon mon.a on smithi006 2024-02-26T06:18:27.989 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:27.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 -- 172.21.15.6:0/2760314696 shutdown_connections 2024-02-26T06:18:27.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 --2- 172.21.15.6:0/2760314696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868107960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:27.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 -- 172.21.15.6:0/2760314696 >> 172.21.15.6:0/2760314696 conn(0x7f78680fcfe0 msgr2=0x7f78680ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:27.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 -- 172.21.15.6:0/2760314696 shutdown_connections 2024-02-26T06:18:27.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.988+0000 7f786e200700 1 -- 172.21.15.6:0/2760314696 wait complete. 2024-02-26T06:18:27.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.989+0000 7f786e200700 1 Processor -- start 2024-02-26T06:18:27.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.989+0000 7f786e200700 1 -- start start 2024-02-26T06:18:27.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786e200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:27.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7868199680 0x7f786819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786e200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f786819df50 0x7f786819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786e200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7868199d70 con 0x7f7868107580 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681a5cc0 con 0x7f7868199680 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786e200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681a5e00 con 0x7f786819df50 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53738/0 (socket says 172.21.15.6:53738) 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786ce00700 1 -- 172.21.15.6:0/1973766513 learned_addr learned my addr 172.21.15.6:0/1973766513 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f785fe00700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7868199680 0x7f786819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.990+0000 7f786d800700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f786819df50 0x7f786819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786ce00700 1 -- 172.21.15.6:0/1973766513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f786819df50 msgr2=0x7f786819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786ce00700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f786819df50 0x7f786819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786ce00700 1 -- 172.21.15.6:0/1973766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7868199680 msgr2=0x7f786819da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786ce00700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7868199680 0x7f786819da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:27.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786ce00700 1 -- 172.21.15.6:0/1973766513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580095f0 con 0x7f7868107580 2024-02-26T06:18:27.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786ce00700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868199140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7858005b40 tx=0x7f785800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:27.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785801d070 con 0x7f7868107580 2024-02-26T06:18:27.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f785800fb50 con 0x7f7868107580 2024-02-26T06:18:27.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858022cf0 con 0x7f7868107580 2024-02-26T06:18:27.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.991+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78681a60e0 con 0x7f7868107580 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.992+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78681a66b0 con 0x7f7868107580 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.993+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f785800fcc0 con 0x7f7868107580 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.993+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78680632a0 con 0x7f7868107580 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.993+0000 7f7866000700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f784c039410 0x7f784c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.993+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f785804d120 con 0x7f7868107580 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.994+0000 7f785fe00700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f784c039410 0x7f784c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:27.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:27.994+0000 7f785fe00700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f784c039410 0x7f784c03b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7850009ad0 tx=0x7f7850009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:28.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.000+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f785800e5c0 con 0x7f7868107580 2024-02-26T06:18:28.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.235+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f786810b840 con 0x7f7868107580 2024-02-26T06:18:28.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.235+0000 7f7866000700 1 -- 172.21.15.6:0/1973766513 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f7858027020 con 0x7f7868107580 2024-02-26T06:18:28.236 INFO:teuthology.orchestra.run.smithi006.stdout:# minimal ceph.conf for 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:18:28.236 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-02-26T06:18:28.236 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:18:28.236 INFO:teuthology.orchestra.run.smithi006.stdout: mon_host = [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-02-26T06:18:28.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.238+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f784c039410 msgr2=0x7f784c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:28.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.238+0000 7f786e200700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f784c039410 0x7f784c03b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7850009ad0 tx=0x7f7850009210 comp rx=0 tx=0).stop 2024-02-26T06:18:28.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.238+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 msgr2=0x7f7868199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:28.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.238+0000 7f786e200700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868199140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7858005b40 tx=0x7f785800bcf0 comp rx=0 tx=0).stop 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 shutdown_connections 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f784c039410 0x7f784c03b8d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f786819df50 0x7f786819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7868107580 0x7f7868199140 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 --2- 172.21.15.6:0/1973766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7868199680 0x7f786819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 >> 172.21.15.6:0/1973766513 conn(0x7f78680fcfe0 msgr2=0x7f7868073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 shutdown_connections 2024-02-26T06:18:28.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:28.239+0000 7f786e200700 1 -- 172.21.15.6:0/1973766513 wait complete. 2024-02-26T06:18:28.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:27 smithi167 ceph-mon[36229]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:28.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:27 smithi167 ceph-mon[36229]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T06:18:28.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:27 smithi167 ceph-mon[36229]: Reconfiguring daemon mon.a on smithi006 2024-02-26T06:18:28.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:27 smithi090 ceph-mon[35681]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:27 smithi090 ceph-mon[35681]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T06:18:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:27 smithi090 ceph-mon[35681]: Reconfiguring daemon mon.a on smithi006 2024-02-26T06:18:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:29.060 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:28 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1973766513' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:29.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:28 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1973766513' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:29.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:28 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1973766513' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:29.636 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-26T06:18:29.637 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:18:29.637 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T06:18:29.725 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:18:29.725 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:18:29.843 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:18:29.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T06:18:29.873 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:18:29.873 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:18:29.942 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:18:29.942 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T06:18:29.976 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:18:29.976 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:18:30.047 INFO:tasks.cephadm:Adding mgr.x on smithi006 2024-02-26T06:18:30.047 INFO:tasks.cephadm:Adding mgr.y on smithi090 2024-02-26T06:18:30.047 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-02-26T06:18:30.048 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch apply mgr '3;smithi006=x;smithi090=y;smithi167=z' 2024-02-26T06:18:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:29 smithi090 ceph-mon[35681]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:29 smithi006 ceph-mon[31790]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:30.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:29 smithi167 ceph-mon[36229]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:30.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:30.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:30.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:30.322 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:18:30.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: Reconfiguring mon.b (monmap changed)... 2024-02-26T06:18:30.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: Reconfiguring daemon mon.b on smithi090 2024-02-26T06:18:30.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:30.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:30.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:30.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: Reconfiguring mon.b (monmap changed)... 2024-02-26T06:18:31.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: Reconfiguring daemon mon.b on smithi090 2024-02-26T06:18:31.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:31.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:31.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:31.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:31.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: Reconfiguring mon.b (monmap changed)... 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: Reconfiguring daemon mon.b on smithi090 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T06:18:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:31.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:31 smithi167 ceph-mon[36229]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:31 smithi167 ceph-mon[36229]: Reconfiguring mon.c (monmap changed)... 2024-02-26T06:18:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:31 smithi167 ceph-mon[36229]: Reconfiguring daemon mon.c on smithi167 2024-02-26T06:18:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:31 smithi006 ceph-mon[31790]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:31 smithi006 ceph-mon[31790]: Reconfiguring mon.c (monmap changed)... 2024-02-26T06:18:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:31 smithi006 ceph-mon[31790]: Reconfiguring daemon mon.c on smithi167 2024-02-26T06:18:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:31 smithi090 ceph-mon[35681]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:31 smithi090 ceph-mon[35681]: Reconfiguring mon.c (monmap changed)... 2024-02-26T06:18:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:31 smithi090 ceph-mon[35681]: Reconfiguring daemon mon.c on smithi167 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 -- 172.21.15.167:0/455331491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8102720 msgr2=0x7f9ab8104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 --2- 172.21.15.167:0/455331491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8102720 0x7f9ab8104b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009990 tx=0x7f9aa8009ca0 comp rx=0 tx=0).stop 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 -- 172.21.15.167:0/455331491 shutdown_connections 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 --2- 172.21.15.167:0/455331491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab8107980 0x7f9ab8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 --2- 172.21.15.167:0/455331491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8102720 0x7f9ab8104b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 --2- 172.21.15.167:0/455331491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab8105050 0x7f9ab8107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 -- 172.21.15.167:0/455331491 >> 172.21.15.167:0/455331491 conn(0x7f9ab80fc130 msgr2=0x7f9ab80fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:32.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 -- 172.21.15.167:0/455331491 shutdown_connections 2024-02-26T06:18:32.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 -- 172.21.15.167:0/455331491 wait complete. 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.841+0000 7f9abcd11700 1 Processor -- start 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 -- start start 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab8102720 0x7f9ab8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab8105050 0x7f9ab8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 0x7f9ab819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab819f840 con 0x7f9ab8105050 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab819f980 con 0x7f9ab8102720 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9abcd11700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab819fb40 con 0x7f9ab8107980 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 0x7f9ab819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 0x7f9ab819f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41200/0 (socket says 172.21.15.167:41200) 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 -- 172.21.15.167:0/396368075 learned_addr learned my addr 172.21.15.167:0/396368075 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:18:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab659c700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab8102720 0x7f9ab8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 -- 172.21.15.167:0/396368075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab8102720 msgr2=0x7f9ab8197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab8102720 0x7f9ab8197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 -- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab8105050 msgr2=0x7f9ab8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab8105050 0x7f9ab8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.842+0000 7f9ab6d9d700 1 -- 172.21.15.167:0/396368075 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80095f0 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.843+0000 7f9ab6d9d700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 0x7f9ab819f270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b9e0 tx=0x7f9ab000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.843+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000c930 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.843+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0014440 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.843+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab819fe20 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.843+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00125a0 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.843+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab81a02b0 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.844+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f9ab0012700 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.844+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8068930 con 0x7f9ab8107980 2024-02-26T06:18:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.844+0000 7f9aaffff700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9aa40391c0 0x7f9aa403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:32.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.844+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ab0006e40 con 0x7f9ab8107980 2024-02-26T06:18:32.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.844+0000 7f9ab659c700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9aa40391c0 0x7f9aa403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:32.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.845+0000 7f9ab659c700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9aa40391c0 0x7f9aa403b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8003790 tx=0x7f9aa8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:32.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:32.849+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ab0013c70 con 0x7f9ab8107980 2024-02-26T06:18:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.052+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi090=y;smithi167=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ab802d910 con 0x7f9aa40391c0 2024-02-26T06:18:33.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.065+0000 7f9aaffff700 1 -- 172.21.15.167:0/396368075 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9ab802d910 con 0x7f9aa40391c0 2024-02-26T06:18:33.065 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-02-26T06:18:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9aa40391c0 msgr2=0x7f9aa403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9aa40391c0 0x7f9aa403b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8003790 tx=0x7f9aa8005fd0 comp rx=0 tx=0).stop 2024-02-26T06:18:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 msgr2=0x7f9ab819f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 0x7f9ab819f270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b9e0 tx=0x7f9ab000bcf0 comp rx=0 tx=0).stop 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 shutdown_connections 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9aa40391c0 0x7f9aa403b680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ab8107980 0x7f9ab819f270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab8102720 0x7f9ab8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 --2- 172.21.15.167:0/396368075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ab8105050 0x7f9ab8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 >> 172.21.15.167:0/396368075 conn(0x7f9ab80fc130 msgr2=0x7f9ab8103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 shutdown_connections 2024-02-26T06:18:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:18:33.067+0000 7f9abcd11700 1 -- 172.21.15.167:0/396368075 wait complete. 2024-02-26T06:18:33.654 DEBUG:teuthology.orchestra.run.smithi090:mgr.y> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.y.service 2024-02-26T06:18:33.657 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.z.service 2024-02-26T06:18:33.659 INFO:tasks.cephadm:Deploying OSDs... 2024-02-26T06:18:33.660 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:18:33.660 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:33.684 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:33.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.722 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T06:18:33.722 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-02-26T06:18:33.775 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-02-26 06:17:29.233393589 +0000 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-02-26 06:08:07.366944572 +0000 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-02-26 06:08:07.366944572 +0000 2024-02-26T06:18:33.776 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-02-26 06:08:07.366944572 +0000 2024-02-26T06:18:33.776 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T06:18:33.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:33.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:33.864 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:33.872 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-02-26T06:18:33.872 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-02-26T06:18:33.872 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000370623 s, 1.4 MB/s 2024-02-26T06:18:33.876 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T06:18:33.933 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-02-26T06:18:33.988 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T06:18:33.989 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:33.989 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-02-26T06:18:33.989 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:33.989 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:33.989 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-02-26 06:17:29.283393038 +0000 2024-02-26T06:18:33.989 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-02-26 06:08:07.954938374 +0000 2024-02-26T06:18:33.990 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-02-26 06:08:07.954938374 +0000 2024-02-26T06:18:33.990 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-02-26 06:08:07.954938374 +0000 2024-02-26T06:18:33.990 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T06:18:34.076 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-02-26T06:18:34.076 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-02-26T06:18:34.076 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.00030416 s, 1.7 MB/s 2024-02-26T06:18:34.081 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T06:18:34.140 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-02-26T06:18:34.194 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T06:18:34.194 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:34.194 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-02-26T06:18:34.195 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:34.195 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:34.195 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-02-26 06:17:29.330392519 +0000 2024-02-26T06:18:34.195 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-02-26 06:08:08.557932018 +0000 2024-02-26T06:18:34.195 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-02-26 06:08:08.557932018 +0000 2024-02-26T06:18:34.195 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-02-26 06:08:08.557932018 +0000 2024-02-26T06:18:34.195 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T06:18:34.293 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-02-26T06:18:34.293 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-02-26T06:18:34.293 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000364934 s, 1.4 MB/s 2024-02-26T06:18:34.300 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T06:18:34.363 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-02-26 06:17:29.377392001 +0000 2024-02-26T06:18:34.417 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-02-26 06:08:09.154925726 +0000 2024-02-26T06:18:34.418 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-02-26 06:08:09.154925726 +0000 2024-02-26T06:18:34.418 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-02-26 06:08:09.154925726 +0000 2024-02-26T06:18:34.418 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T06:18:34.509 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-02-26T06:18:34.509 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-02-26T06:18:34.509 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000474586 s, 1.1 MB/s 2024-02-26T06:18:34.513 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T06:18:34.571 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-02-26T06:18:34.571 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T06:18:34.590 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T06:18:34.590 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-02-26T06:18:34.647 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T06:18:34.647 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-02-26 06:18:17.462321766 +0000 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-02-26 06:08:09.403668033 +0000 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-02-26 06:08:09.403668033 +0000 2024-02-26T06:18:34.648 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-02-26 06:08:09.403668033 +0000 2024-02-26T06:18:34.648 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T06:18:34.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:34 smithi006 ceph-mon[31790]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi090=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:34.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:34 smithi006 ceph-mon[31790]: Saving service mgr spec with placement smithi006=x;smithi090=y;smithi167=z;count:3 2024-02-26T06:18:34.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:34 smithi006 ceph-mon[31790]: Deploying daemon mgr.z on smithi167 2024-02-26T06:18:34.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:34 smithi090 ceph-mon[35681]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi090=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:34.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:34 smithi090 ceph-mon[35681]: Saving service mgr spec with placement smithi006=x;smithi090=y;smithi167=z;count:3 2024-02-26T06:18:34.715 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:34 smithi090 ceph-mon[35681]: Deploying daemon mgr.z on smithi167 2024-02-26T06:18:34.717 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-02-26T06:18:34.718 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-02-26T06:18:34.718 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000272327 s, 1.9 MB/s 2024-02-26T06:18:34.719 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T06:18:34.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:34 smithi167 ceph-mon[36229]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi090=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:34.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:34 smithi167 ceph-mon[36229]: Saving service mgr spec with placement smithi006=x;smithi090=y;smithi167=z;count:3 2024-02-26T06:18:34.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:34 smithi167 ceph-mon[36229]: Deploying daemon mgr.z on smithi167 2024-02-26T06:18:34.777 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-02-26T06:18:34.833 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T06:18:34.833 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:34.833 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-02-26T06:18:34.833 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:34.833 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:34.834 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-02-26 06:18:17.463321749 +0000 2024-02-26T06:18:34.834 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-02-26 06:08:09.747662357 +0000 2024-02-26T06:18:34.834 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-02-26 06:08:09.747662357 +0000 2024-02-26T06:18:34.834 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-02-26 06:08:09.747662357 +0000 2024-02-26T06:18:34.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T06:18:34.900 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-02-26T06:18:34.900 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-02-26T06:18:34.900 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000205851 s, 2.5 MB/s 2024-02-26T06:18:34.901 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T06:18:34.957 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-02-26 06:18:17.463321749 +0000 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-02-26 06:08:10.095656615 +0000 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-02-26 06:08:10.095656615 +0000 2024-02-26T06:18:35.013 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-02-26 06:08:10.095656615 +0000 2024-02-26T06:18:35.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T06:18:35.079 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-02-26T06:18:35.080 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-02-26T06:18:35.080 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000201137 s, 2.5 MB/s 2024-02-26T06:18:35.081 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T06:18:35.137 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-02-26 06:18:17.463321749 +0000 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-02-26 06:08:10.452650725 +0000 2024-02-26T06:18:35.193 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-02-26 06:08:10.452650725 +0000 2024-02-26T06:18:35.194 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-02-26 06:08:10.452650725 +0000 2024-02-26T06:18:35.194 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T06:18:35.259 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-02-26T06:18:35.259 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-02-26T06:18:35.260 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000185277 s, 2.8 MB/s 2024-02-26T06:18:35.261 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T06:18:35.317 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-02-26T06:18:35.317 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T06:18:35.334 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T06:18:35.334 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-02-26 06:18:35.175883823 +0000 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-02-26 06:07:04.628742982 +0000 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-02-26 06:07:04.628742982 +0000 2024-02-26T06:18:35.389 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-02-26 06:07:04.628742982 +0000 2024-02-26T06:18:35.390 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T06:18:35.437 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 systemd[1]: Starting Ceph mgr.z for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:18:35.456 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-02-26T06:18:35.457 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-02-26T06:18:35.457 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000195076 s, 2.6 MB/s 2024-02-26T06:18:35.458 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T06:18:35.513 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-02-26T06:18:35.568 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T06:18:35.568 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:35.568 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-26T06:18:35.568 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:35.568 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:35.569 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-02-26 06:18:35.175883823 +0000 2024-02-26T06:18:35.569 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-02-26 06:07:04.986736404 +0000 2024-02-26T06:18:35.569 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-02-26 06:07:04.986736404 +0000 2024-02-26T06:18:35.569 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-02-26 06:07:04.986736404 +0000 2024-02-26T06:18:35.569 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T06:18:35.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:35 smithi090 ceph-mon[35681]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-02-26T06:18:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-02-26T06:18:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.0143842 s, 35.6 kB/s 2024-02-26T06:18:35.654 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T06:18:35.710 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-02-26T06:18:35.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:35 smithi167 ceph-mon[36229]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:35.737 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 podman[37895]: 2024-02-26 06:18:35.360784483 +0000 UTC m=+0.018108878 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:18:35.738 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 podman[37895]: 2024-02-26 06:18:35.48592969 +0000 UTC m=+0.143254072 container create f4db46f9f89bbfbfd2250f70f6ef1a31ff73d3a74612367ceeaec8764b28ef82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-26T06:18:35.738 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 podman[37895]: 2024-02-26 06:18:35.738441797 +0000 UTC m=+0.395766199 container init f4db46f9f89bbfbfd2250f70f6ef1a31ff73d3a74612367ceeaec8764b28ef82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0) 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-02-26 06:18:35.175883823 +0000 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-02-26 06:07:05.350729716 +0000 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-02-26 06:07:05.350729716 +0000 2024-02-26T06:18:35.751 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-02-26 06:07:05.350729716 +0000 2024-02-26T06:18:35.751 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T06:18:35.817 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-02-26T06:18:35.818 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-02-26T06:18:35.818 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000175768 s, 2.9 MB/s 2024-02-26T06:18:35.819 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T06:18:35.876 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-02-26T06:18:35.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:35 smithi006 ceph-mon[31790]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:35.930 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-02-26 06:18:35.175883823 +0000 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-02-26 06:07:05.702723248 +0000 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-02-26 06:07:05.702723248 +0000 2024-02-26T06:18:35.931 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-02-26 06:07:05.702723248 +0000 2024-02-26T06:18:35.931 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T06:18:35.994 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 podman[37895]: 2024-02-26 06:18:35.741187182 +0000 UTC m=+0.398511564 container start f4db46f9f89bbfbfd2250f70f6ef1a31ff73d3a74612367ceeaec8764b28ef82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-26T06:18:35.994 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 ceph-mgr[38021]: -- 172.21.15.167:0/486885167 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dbacce81a0 con 0x55dbaccbf000 2024-02-26T06:18:35.994 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 bash[37895]: f4db46f9f89bbfbfd2250f70f6ef1a31ff73d3a74612367ceeaec8764b28ef82 2024-02-26T06:18:35.995 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:35 smithi167 systemd[1]: Started Ceph mgr.z for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:18:35.997 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-02-26T06:18:35.997 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-02-26T06:18:35.997 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000194022 s, 2.6 MB/s 2024-02-26T06:18:35.998 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T06:18:36.054 INFO:tasks.cephadm:Deploying osd.0 on smithi006 with /dev/vg_nvme/lv_4... 2024-02-26T06:18:36.054 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T06:18:36.509 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:18:37.092 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:36.294+0000 7f858fa1e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:18:37.092 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:36.387+0000 7f858fa1e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:18:37.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:37.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:18:37.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:37.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:37.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:37.244 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.028+0000 7f858fa1e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:18:37.244 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.123+0000 7f858fa1e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:37.733 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.270+0000 7f858fa1e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:18:37.733 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.369+0000 7f858fa1e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:18:37.734 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.458+0000 7f858fa1e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:18:37.909 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:37 smithi090 systemd[1]: Starting Ceph mgr.y for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:18:37.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-mon[36229]: Deploying daemon mgr.y on smithi090 2024-02-26T06:18:37.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-mon[36229]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:37.995 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.733+0000 7f858fa1e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:18:37.995 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.845+0000 7f858fa1e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:18:37.995 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:37.933+0000 7f858fa1e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:18:38.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:37 smithi006 ceph-mon[31790]: Deploying daemon mgr.y on smithi090 2024-02-26T06:18:38.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:37 smithi006 ceph-mon[31790]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:38.216 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:37 smithi090 ceph-mon[35681]: Deploying daemon mgr.y on smithi090 2024-02-26T06:18:38.216 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:37 smithi090 ceph-mon[35681]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:38.217 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 podman[37093]: 2024-02-26 06:18:38.005038953 +0000 UTC m=+0.098985376 container create 8bed9c8b972df0354b50ac446bef9995ddec47f8c49bfb2f8191ee302dfd94d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , ceph=True) 2024-02-26T06:18:38.217 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 podman[37093]: 2024-02-26 06:18:37.923827497 +0000 UTC m=+0.017773931 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:18:38.217 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 podman[37093]: 2024-02-26 06:18:38.217436712 +0000 UTC m=+0.311383155 container init 8bed9c8b972df0354b50ac446bef9995ddec47f8c49bfb2f8191ee302dfd94d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-26T06:18:38.302 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:38.030+0000 7f858fa1e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:18:38.302 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:38.210+0000 7f858fa1e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:18:38.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 podman[37093]: 2024-02-26 06:18:38.220196426 +0000 UTC m=+0.314142857 container start 8bed9c8b972df0354b50ac446bef9995ddec47f8c49bfb2f8191ee302dfd94d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:18:38.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 ceph-mgr[37112]: -- 172.21.15.90:0/4001021662 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c085c661a0 con 0x55c085c3cc00 2024-02-26T06:18:38.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 bash[37093]: 8bed9c8b972df0354b50ac446bef9995ddec47f8c49bfb2f8191ee302dfd94d2 2024-02-26T06:18:38.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 systemd[1]: Started Ceph mgr.y for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:18:39.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:38.762+0000 7f3e4c0fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:18:39.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:38.859+0000 7f3e4c0fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:18:39.127 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T06:18:39.127 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T06:18:39.127 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-02-26T06:18:39.127 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-02-26T06:18:39.127 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0386327 s, 271 MB/s 2024-02-26T06:18:39.127 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-02-26T06:18:39.494 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:39.220+0000 7f858fa1e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:39.361+0000 7f858fa1e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-mon[36229]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:39.657 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-mon[35681]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:39.542+0000 7f3e4c0fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:18:39.658 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:39.639+0000 7f3e4c0fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:18:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:39 smithi006 ceph-mon[31790]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:39.797 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:39.706+0000 7f858fa1e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:18:39.925 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_4 2024-02-26T06:18:39.964 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:39.782+0000 7f3e4c0fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:18:39.964 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:39.877+0000 7f3e4c0fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:18:39.964 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:39.964+0000 7f3e4c0fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:18:40.181 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:39.797+0000 7f858fa1e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:18:40.181 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:39.996+0000 7f858fa1e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:18:40.181 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:40.180+0000 7f858fa1e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:18:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:18:40.519 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:40 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:40.232+0000 7f3e4c0fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:18:40.519 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:40 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:40.341+0000 7f3e4c0fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:18:40.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:40 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:40.426+0000 7f3e4c0fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:18:40.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:40 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:40.519+0000 7f3e4c0fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:18:40.725 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:40.464+0000 7f858fa1e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:18:40.865 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:40 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:40.693+0000 7f3e4c0fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:18:40.994 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:40.724+0000 7f858fa1e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:18:40.995 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:40.958+0000 7f858fa1e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:18:41.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:41.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:41.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:41 smithi006 ceph-mon[31790]: Reconfiguring daemon mgr.x on smithi006 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:41.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T06:18:41.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:41.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:41.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:41.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-mon[35681]: Reconfiguring daemon mgr.x on smithi006 2024-02-26T06:18:41.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:41.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:41.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-mon[36229]: Reconfiguring daemon mgr.x on smithi006 2024-02-26T06:18:42.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:41.656+0000 7f3e4c0fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:18:42.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:41 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:41.785+0000 7f3e4c0fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:18:42.244 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:41.748+0000 7f858fa1e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:18:42.399 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:42 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:42.121+0000 7f3e4c0fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:18:42.399 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:42 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:42.208+0000 7f3e4c0fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:18:42.399 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:42 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:42.399+0000 7f3e4c0fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:18:42.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 -- 172.21.15.6:0/1636630471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890103840 msgr2=0x7fb89010a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:42.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 --2- 172.21.15.6:0/1636630471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890103840 0x7fb89010a0f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb88800b0d0 tx=0x7fb88800b3e0 comp rx=0 tx=0).stop 2024-02-26T06:18:42.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 -- 172.21.15.6:0/1636630471 shutdown_connections 2024-02-26T06:18:42.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 --2- 172.21.15.6:0/1636630471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890103840 0x7fb89010a0f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:42.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 --2- 172.21.15.6:0/1636630471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb8901020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:42.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 --2- 172.21.15.6:0/1636630471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb890102ea0 0x7fb890103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:42.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 -- 172.21.15.6:0/1636630471 >> 172.21.15.6:0/1636630471 conn(0x7fb8900fd280 msgr2=0x7fb8900ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:18:42.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.488+0000 7fb895400700 1 -- 172.21.15.6:0/1636630471 shutdown_connections 2024-02-26T06:18:42.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.489+0000 7fb895400700 1 -- 172.21.15.6:0/1636630471 wait complete. 2024-02-26T06:18:42.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 Processor -- start 2024-02-26T06:18:42.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 -- start start 2024-02-26T06:18:42.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb890197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890102ea0 0x7fb890198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb890103840 0x7fb89019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89019f970 con 0x7fb890103840 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89019fab0 con 0x7fb890102ea0 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.490+0000 7fb895400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89019fc40 con 0x7fb890101ca0 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb890197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb890197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:44724/0 (socket says 172.21.15.6:44724) 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 -- 172.21.15.6:0/3957712167 learned_addr learned my addr 172.21.15.6:0/3957712167 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88de00700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890102ea0 0x7fb890198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88f200700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb890103840 0x7fb89019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:42.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 -- 172.21.15.6:0/3957712167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890102ea0 msgr2=0x7fb890198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890102ea0 0x7fb890198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 -- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb890103840 msgr2=0x7fb89019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb890103840 0x7fb89019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 -- 172.21.15.6:0/3957712167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888009d00 con 0x7fb890101ca0 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.491+0000 7fb88e800700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb890197de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9a0 tx=0x7fb88000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.492+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009790 con 0x7fb890101ca0 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.492+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb880004500 con 0x7fb890101ca0 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.492+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89019ff20 con 0x7fb890101ca0 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.492+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880003e30 con 0x7fb890101ca0 2024-02-26T06:18:42.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.492+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901a0340 con 0x7fb890101ca0 2024-02-26T06:18:42.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.493+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb890068930 con 0x7fb890101ca0 2024-02-26T06:18:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.493+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fb880018430 con 0x7fb890101ca0 2024-02-26T06:18:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.494+0000 7fb887400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb8740391c0 0x7fb87403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:18:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.494+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb880014070 con 0x7fb890101ca0 2024-02-26T06:18:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.494+0000 7fb88de00700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb8740391c0 0x7fb87403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:18:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.495+0000 7fb88de00700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb8740391c0 0x7fb87403b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009a10 tx=0x7fb87c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:18:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.499+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb880051020 con 0x7fb890101ca0 2024-02-26T06:18:42.626 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:42.318+0000 7f858fa1e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:18:42.626 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:42.415+0000 7f858fa1e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:18:42.626 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:18:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:18:42.626+0000 7f858fa1e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:18:42.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:42.744+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb89002d990 con 0x7fb8740391c0 2024-02-26T06:18:42.856 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:42 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:42.579+0000 7f3e4c0fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:18:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:43.058+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 14) v1 ==== 91349+0+0 (secure 0 0 0) 0x7fb88000f3c0 con 0x7fb890101ca0 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:42 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:42.856+0000 7f3e4c0fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: Standby manager daemon z started 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:18:43.113 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:18:43.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:18:43.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:43.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:43.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:43.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:43.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:43.113+0000 7f3e4c0fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:18:43.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:43 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:43.345+0000 7f3e4c0fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:18:43.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: Standby manager daemon z started 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:43.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:43.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:43.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: Standby manager daemon z started 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/4138124415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:18:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:44.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:44.170+0000 7f3e4c0fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:18:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: mgrmap e14: x(active, since 91s), standbys: z 2024-02-26T06:18:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:18:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:44.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:44.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:44.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: mgrmap e14: x(active, since 91s), standbys: z 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:44.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:44 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: mgrmap e14: x(active, since 91s), standbys: z 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:18:44.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:44 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:18:45.042 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:44.735+0000 7f3e4c0fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:18:45.043 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:44.831+0000 7f3e4c0fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:18:45.043 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:18:45.042+0000 7f3e4c0fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:18:45.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:18:45.350+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mon.1 v2:172.21.15.167:3300/0 8 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7fb88000fcf0 con 0x7fb890101ca0 2024-02-26T06:18:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-mon[35681]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-mon[35681]: Standby manager daemon y started 2024-02-26T06:18:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:18:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:18:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:18:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:45 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:18:45.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:45 smithi006 ceph-mon[31790]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:45.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:45 smithi006 ceph-mon[31790]: Standby manager daemon y started 2024-02-26T06:18:45.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:45 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:18:45.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:45 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:18:45.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:45 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:18:45.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:45 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:18:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:45 smithi167 ceph-mon[36229]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:45 smithi167 ceph-mon[36229]: Standby manager daemon y started 2024-02-26T06:18:45.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:45 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:18:45.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:45 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:18:45.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:45 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:18:45.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:45 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/2152629118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:18:46.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:46 smithi006 ceph-mon[31790]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-02-26T06:18:46.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:18:46.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:46 smithi167 ceph-mon[36229]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-02-26T06:18:46.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:18:46.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:46 smithi090 ceph-mon[35681]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-02-26T06:18:46.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:18:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:47 smithi006 ceph-mon[31790]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:47 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2057848137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]: dispatch 2024-02-26T06:18:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:47 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]: dispatch 2024-02-26T06:18:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:47 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]': finished 2024-02-26T06:18:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:47 smithi006 ceph-mon[31790]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T06:18:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:18:47.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:47 smithi167 ceph-mon[36229]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:47.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:47 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2057848137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]: dispatch 2024-02-26T06:18:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:47 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]: dispatch 2024-02-26T06:18:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:47 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]': finished 2024-02-26T06:18:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:47 smithi167 ceph-mon[36229]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T06:18:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:18:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:47 smithi090 ceph-mon[35681]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:47 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2057848137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]: dispatch 2024-02-26T06:18:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:47 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]: dispatch 2024-02-26T06:18:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:47 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d33ce97-4574-4b39-a887-734262e13d40"}]': finished 2024-02-26T06:18:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:47 smithi090 ceph-mon[35681]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T06:18:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:18:48.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:48 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2163724432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:48.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:48 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2163724432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:48.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:48 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2163724432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:18:49.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:49 smithi006 ceph-mon[31790]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:49.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:49 smithi167 ceph-mon[36229]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:49.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:49 smithi090 ceph-mon[35681]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:51.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:51 smithi006 ceph-mon[31790]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:51 smithi167 ceph-mon[36229]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:51 smithi090 ceph-mon[35681]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:53.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:53 smithi006 ceph-mon[31790]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:53.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:53 smithi167 ceph-mon[36229]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:53.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:53 smithi090 ceph-mon[35681]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:55.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:55 smithi006 ceph-mon[31790]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:55.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:55 smithi167 ceph-mon[36229]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:55.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:55 smithi090 ceph-mon[35681]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:56.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:56 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T06:18:56.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:56 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:56.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:56 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T06:18:56.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:56 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:56.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:56 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T06:18:56.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:56 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:18:57.521 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:57 smithi006 ceph-mon[31790]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:57.521 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:57 smithi006 ceph-mon[31790]: Deploying daemon osd.0 on smithi006 2024-02-26T06:18:57.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:57 smithi167 ceph-mon[36229]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:57.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:57 smithi167 ceph-mon[36229]: Deploying daemon osd.0 on smithi006 2024-02-26T06:18:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:57 smithi090 ceph-mon[35681]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:57 smithi090 ceph-mon[35681]: Deploying daemon osd.0 on smithi006 2024-02-26T06:18:59.642 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:18:59 smithi006 ceph-mon[31790]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:59.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:18:59 smithi167 ceph-mon[36229]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:18:59.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:18:59 smithi090 ceph-mon[35681]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:01.663 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:01 smithi006 ceph-mon[31790]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:01.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:01 smithi167 ceph-mon[36229]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:01.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:01 smithi090 ceph-mon[35681]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:03.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:03 smithi006 ceph-mon[31790]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:03.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:03 smithi167 ceph-mon[36229]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:03.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:03 smithi090 ceph-mon[35681]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:05.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:05 smithi006 ceph-mon[31790]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:05 smithi167 ceph-mon[36229]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:05.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:05 smithi090 ceph-mon[35681]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:05.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:05.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:05.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:07.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:07 smithi090 ceph-mon[35681]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:07.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:07 smithi006 ceph-mon[31790]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:07.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:07 smithi167 ceph-mon[36229]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:08.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:08.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:08.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:08 smithi006 ceph-mon[31790]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:19:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:08.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:08.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:08 smithi167 ceph-mon[36229]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:19:08.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:08.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:08.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:08.869 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:08 smithi090 ceph-mon[35681]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T06:19:09.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.048+0000 7fb887400700 1 -- 172.21.15.6:0/3957712167 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb89002d990 con 0x7fb8740391c0 2024-02-26T06:19:09.048 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 0 on host 'smithi006' 2024-02-26T06:19:09.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb8740391c0 msgr2=0x7fb87403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb8740391c0 0x7fb87403b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009a10 tx=0x7fb87c009190 comp rx=0 tx=0).stop 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 msgr2=0x7fb890197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb890197de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9a0 tx=0x7fb88000ecb0 comp rx=0 tx=0).stop 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 shutdown_connections 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb8740391c0 0x7fb87403b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb890103840 0x7fb89019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb890101ca0 0x7fb890197de0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 --2- 172.21.15.6:0/3957712167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb890102ea0 0x7fb890198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 >> 172.21.15.6:0/3957712167 conn(0x7fb8900fd280 msgr2=0x7fb8901061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 shutdown_connections 2024-02-26T06:19:09.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:19:09.051+0000 7fb895400700 1 -- 172.21.15.6:0/3957712167 wait complete. 2024-02-26T06:19:09.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T06:19:09.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T06:19:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:19:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:09.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:09.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:09.898 DEBUG:teuthology.orchestra.run.smithi006:osd.0> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.0.service 2024-02-26T06:19:09.904 INFO:tasks.cephadm:Deploying osd.1 on smithi090 with /dev/vg_nvme/lv_4... 2024-02-26T06:19:09.904 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T06:19:10.146 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:19:10.543 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:10 smithi090 ceph-mon[35681]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:19:10.543 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:10 smithi090 ceph-mon[35681]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T06:19:10.544 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:10 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:10.544 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:10 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:10.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:10 smithi006 ceph-mon[31790]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:19:10.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:10 smithi006 ceph-mon[31790]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T06:19:10.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:10 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:10.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:10 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:10.673 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:19:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0[46314]: 2024-02-26T06:19:10.521+0000 7fc282c00700 -1 osd.0 0 waiting for initial osdmap 2024-02-26T06:19:10.673 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:19:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0[46314]: 2024-02-26T06:19:10.539+0000 7fc279400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:19:10.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:10 smithi167 ceph-mon[36229]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:19:10.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:10 smithi167 ceph-mon[36229]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T06:19:10.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:10 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:10.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:10 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:11.619 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:11 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:19:11.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:11 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:19:11.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:11 smithi006 ceph-mon[31790]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:11.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:11.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:11 smithi006 ceph-mon[31790]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' 2024-02-26T06:19:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:11 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:19:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:11 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:19:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:11 smithi167 ceph-mon[36229]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:11.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:11 smithi167 ceph-mon[36229]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' 2024-02-26T06:19:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:11 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:19:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:11 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:19:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:11 smithi090 ceph-mon[35681]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:11.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:11.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:11 smithi090 ceph-mon[35681]: from='osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447]' entity='osd.0' 2024-02-26T06:19:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T06:19:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T06:19:11.926 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-02-26T06:19:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-02-26T06:19:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113553 s, 923 MB/s 2024-02-26T06:19:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-02-26T06:19:12.451 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-02-26T06:19:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:12 smithi090 ceph-mon[35681]: osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] boot 2024-02-26T06:19:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:12 smithi090 ceph-mon[35681]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T06:19:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:19:12.730 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:12.731 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:12 smithi006 ceph-mon[31790]: osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] boot 2024-02-26T06:19:12.731 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:12 smithi006 ceph-mon[31790]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T06:19:12.731 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:12 smithi167 ceph-mon[36229]: osd.0 [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] boot 2024-02-26T06:19:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:12 smithi167 ceph-mon[36229]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T06:19:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:19:13.830 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:13 smithi006 ceph-mon[31790]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:13.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:13 smithi006 ceph-mon[31790]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T06:19:13.845 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:13 smithi090 ceph-mon[35681]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:13.845 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:13 smithi090 ceph-mon[35681]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T06:19:13.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:13 smithi167 ceph-mon[36229]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T06:19:13.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:13 smithi167 ceph-mon[36229]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 -- 172.21.15.90:0/214168900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 msgr2=0x7fc244108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 --2- 172.21.15.90:0/214168900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc244108fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b0d0 tx=0x7fc23c00b3e0 comp rx=0 tx=0).stop 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 -- 172.21.15.90:0/214168900 shutdown_connections 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 --2- 172.21.15.90:0/214168900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc244108fe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 --2- 172.21.15.90:0/214168900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244100b90 0x7fc244100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 --2- 172.21.15.90:0/214168900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc244101d90 0x7fc2441021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 -- 172.21.15.90:0/214168900 >> 172.21.15.90:0/214168900 conn(0x7fc2440fc130 msgr2=0x7fc2440fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:14.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 -- 172.21.15.90:0/214168900 shutdown_connections 2024-02-26T06:19:14.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.047+0000 7fc24a435700 1 -- 172.21.15.90:0/214168900 wait complete. 2024-02-26T06:19:14.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 Processor -- start 2024-02-26T06:19:14.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 -- start start 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc244100b90 0x7fc244197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244101d90 0x7fc244198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc24419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24419f870 con 0x7fc244100b90 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24419f9b0 con 0x7fc244102730 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc24a435700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24419fb70 con 0x7fc244101d90 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc2489d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc24419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc2489d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc24419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53712/0 (socket says 172.21.15.90:53712) 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.048+0000 7fc2489d2700 1 -- 172.21.15.90:0/236612635 learned_addr learned my addr 172.21.15.90:0/236612635 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2437fe700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244101d90 0x7fc244198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:14.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2489d2700 1 -- 172.21.15.90:0/236612635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244101d90 msgr2=0x7fc244198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2489d2700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244101d90 0x7fc244198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2489d2700 1 -- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc244100b90 msgr2=0x7fc244197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2489d2700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc244100b90 0x7fc244197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2489d2700 1 -- 172.21.15.90:0/236612635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c009d00 con 0x7fc244102730 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2489d2700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc24419f2a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc23c007c90 tx=0x7fc23c009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00e040 con 0x7fc244102730 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24419fdf0 con 0x7fc244102730 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c004e60 con 0x7fc244102730 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c012bc0 con 0x7fc244102730 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.049+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2441a0230 con 0x7fc244102730 2024-02-26T06:19:14.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.050+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc244107ef0 con 0x7fc244102730 2024-02-26T06:19:14.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.052+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7fc23c012d20 con 0x7fc244102730 2024-02-26T06:19:14.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.052+0000 7fc2417fa700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc22c0a20c0 0x7fc22c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:14.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.052+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fc23c0cdb00 con 0x7fc244102730 2024-02-26T06:19:14.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.053+0000 7fc243fff700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc22c0a20c0 0x7fc22c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:14.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.054+0000 7fc243fff700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc22c0a20c0 0x7fc22c0a4580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc244199200 tx=0x7fc234005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:14.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.056+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc23c0985d0 con 0x7fc244102730 2024-02-26T06:19:14.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:14.258+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc24402d990 con 0x7fc22c0a20c0 2024-02-26T06:19:14.854 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:14.855 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:14.855 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:14.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:14.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:14.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:14.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:14.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:14.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:15.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi006 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi006 to 17095M 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.754 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:15.755 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:15.755 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi006 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi006 to 17095M 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:15.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi006 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi006 to 17095M 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:16.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:16.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:16.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:17.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:17.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/428406219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]: dispatch 2024-02-26T06:19:17.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]: dispatch 2024-02-26T06:19:17.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]': finished 2024-02-26T06:19:17.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T06:19:17.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:17.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:17 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/454826903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:18.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:18.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/428406219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]: dispatch 2024-02-26T06:19:18.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]: dispatch 2024-02-26T06:19:18.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]': finished 2024-02-26T06:19:18.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T06:19:18.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:18.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:17 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/454826903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:18.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:18.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/428406219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]: dispatch 2024-02-26T06:19:18.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]: dispatch 2024-02-26T06:19:18.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71372d37-1d3f-46c1-abc4-3a33d0daf773"}]': finished 2024-02-26T06:19:18.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T06:19:18.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:18.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:17 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/454826903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:19 smithi167 ceph-mon[36229]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:20.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:19 smithi090 ceph-mon[35681]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:20.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:19 smithi006 ceph-mon[31790]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:21 smithi090 ceph-mon[35681]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:21.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:21 smithi167 ceph-mon[36229]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:22.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:21 smithi006 ceph-mon[31790]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:23 smithi090 ceph-mon[35681]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:23.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:23 smithi167 ceph-mon[36229]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:24.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:23 smithi006 ceph-mon[31790]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:24.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T06:19:24.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:25.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T06:19:25.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:25.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T06:19:25.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:25.826 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:25 smithi090 ceph-mon[35681]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:25.827 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:25 smithi090 ceph-mon[35681]: Deploying daemon osd.1 on smithi090 2024-02-26T06:19:25.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:25 smithi167 ceph-mon[36229]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:25.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:25 smithi167 ceph-mon[36229]: Deploying daemon osd.1 on smithi090 2024-02-26T06:19:26.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:25 smithi006 ceph-mon[31790]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:26.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:25 smithi006 ceph-mon[31790]: Deploying daemon osd.1 on smithi090 2024-02-26T06:19:27.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:27 smithi090 ceph-mon[35681]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:27.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:27 smithi167 ceph-mon[36229]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:28.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:27 smithi006 ceph-mon[31790]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:29 smithi090 ceph-mon[35681]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:29.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:29 smithi167 ceph-mon[36229]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:29.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:29.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:29.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:30.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:29 smithi006 ceph-mon[31790]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:30.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:31.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:31 smithi167 ceph-mon[36229]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:31.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:32.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:31 smithi090 ceph-mon[35681]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:32.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:32.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:32.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:32.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:32.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:31 smithi006 ceph-mon[31790]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:32.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:32.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:32.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:32.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:32.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:33.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.005+0000 7fc2417fa700 1 -- 172.21.15.90:0/236612635 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc24402d990 con 0x7fc22c0a20c0 2024-02-26T06:19:33.004 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 1 on host 'smithi090' 2024-02-26T06:19:33.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc22c0a20c0 msgr2=0x7fc22c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc22c0a20c0 0x7fc22c0a4580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc244199200 tx=0x7fc234005d20 comp rx=0 tx=0).stop 2024-02-26T06:19:33.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 msgr2=0x7fc24419f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:33.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc24419f2a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc23c007c90 tx=0x7fc23c009430 comp rx=0 tx=0).stop 2024-02-26T06:19:33.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 shutdown_connections 2024-02-26T06:19:33.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc22c0a20c0 0x7fc22c0a4580 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc244102730 0x7fc24419f2a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc244100b90 0x7fc244197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 --2- 172.21.15.90:0/236612635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244101d90 0x7fc244198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:33.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 >> 172.21.15.90:0/236612635 conn(0x7fc2440fc130 msgr2=0x7fc244105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:33.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 shutdown_connections 2024-02-26T06:19:33.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:19:33.006+0000 7fc24a435700 1 -- 172.21.15.90:0/236612635 wait complete. 2024-02-26T06:19:33.635 DEBUG:teuthology.orchestra.run.smithi090:osd.1> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.1.service 2024-02-26T06:19:33.637 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-02-26T06:19:33.638 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T06:19:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:33 smithi090 ceph-mon[35681]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:33 smithi090 ceph-mon[35681]: from='osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:19:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:33 smithi090 ceph-mon[35681]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:19:33.886 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:19:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:33 smithi167 ceph-mon[36229]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:33 smithi167 ceph-mon[36229]: from='osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:19:33.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:33 smithi167 ceph-mon[36229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:19:34.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:33 smithi006 ceph-mon[31790]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:34.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:34.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:34.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:34.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:33 smithi006 ceph-mon[31790]: from='osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:19:34.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:33 smithi006 ceph-mon[31790]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T06:19:35.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-mon[35681]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T06:19:35.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-mon[35681]: from='osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:19:35.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-mon[35681]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:35.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-mon[35681]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T06:19:35.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:35.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-mon[35681]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:19:35.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:35 smithi167 ceph-mon[36229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T06:19:35.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:35 smithi167 ceph-mon[36229]: from='osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:19:35.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:35 smithi167 ceph-mon[36229]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:35.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:35 smithi167 ceph-mon[36229]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T06:19:35.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:35.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:35 smithi167 ceph-mon[36229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:35 smithi006 ceph-mon[31790]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:35 smithi006 ceph-mon[31790]: from='osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:19:35.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:35 smithi006 ceph-mon[31790]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:35.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:35 smithi006 ceph-mon[31790]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T06:19:35.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:35.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:35 smithi006 ceph-mon[31790]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:19:35.804 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T06:19:35.804 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T06:19:35.804 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-02-26T06:19:35.804 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-02-26T06:19:35.804 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0139293 s, 753 MB/s 2024-02-26T06:19:35.804 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-02-26T06:19:36.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:36 smithi090 ceph-mon[35681]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:19:36.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:36 smithi090 ceph-mon[35681]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T06:19:36.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:36 smithi090 ceph-mon[35681]: from='osd.1 ' entity='osd.1' 2024-02-26T06:19:36.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:36 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1[40326]: 2024-02-26T06:19:35.829+0000 7fb6ac03d700 -1 osd.1 0 waiting for initial osdmap 2024-02-26T06:19:36.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:19:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1[40326]: 2024-02-26T06:19:35.834+0000 7fb6a4629700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:19:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:36 smithi167 ceph-mon[36229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:19:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:36 smithi167 ceph-mon[36229]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T06:19:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:36 smithi167 ceph-mon[36229]: from='osd.1 ' entity='osd.1' 2024-02-26T06:19:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:36 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.420 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-02-26T06:19:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:36 smithi006 ceph-mon[31790]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:19:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:36 smithi006 ceph-mon[31790]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T06:19:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:36 smithi006 ceph-mon[31790]: from='osd.1 ' entity='osd.1' 2024-02-26T06:19:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:36 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:36.666 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:19:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:19:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:19:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi090 2024-02-26T06:19:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi090 to 17080M 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] boot 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T06:19:37.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi090 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi090 to 17080M 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] boot 2024-02-26T06:19:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T06:19:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:37.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:19:37.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:19:37.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi090 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi090 to 17080M 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: osd.1 [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] boot 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T06:19:37.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:19:38.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 -- 172.21.15.167:0/3858433817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc103840 msgr2=0x7faafc10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:38.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 --2- 172.21.15.167:0/3858433817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc103840 0x7faafc10a0f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b0a0 tx=0x7faaf400b3b0 comp rx=0 tx=0).stop 2024-02-26T06:19:38.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 -- 172.21.15.167:0/3858433817 shutdown_connections 2024-02-26T06:19:38.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 --2- 172.21.15.167:0/3858433817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc103840 0x7faafc10a0f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 --2- 172.21.15.167:0/3858433817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc101ca0 0x7faafc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 --2- 172.21.15.167:0/3858433817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faafc102ea0 0x7faafc103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 -- 172.21.15.167:0/3858433817 >> 172.21.15.167:0/3858433817 conn(0x7faafc0fd280 msgr2=0x7faafc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 -- 172.21.15.167:0/3858433817 shutdown_connections 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 -- 172.21.15.167:0/3858433817 wait complete. 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 Processor -- start 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.296+0000 7fab045ae700 1 -- start start 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc101ca0 0x7faafc075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 0x7faafc074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faafc103840 0x7faafc0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc074d20 con 0x7faafc103840 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc074e90 con 0x7faafc101ca0 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc1a1e80 con 0x7faafc102ea0 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 0x7faafc074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:38.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 0x7faafc074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46802/0 (socket says 172.21.15.167:46802) 2024-02-26T06:19:38.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 -- 172.21.15.167:0/2042871621 learned_addr learned my addr 172.21.15.167:0/2042871621 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:19:38.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab0234a700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc101ca0 0x7faafc075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:38.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 -- 172.21.15.167:0/2042871621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc101ca0 msgr2=0x7faafc075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:38.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc101ca0 0x7faafc075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:38.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 -- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faafc103840 msgr2=0x7faafc0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:38.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faafc103840 0x7faafc0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:38.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 -- 172.21.15.167:0/2042871621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf4009d00 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab01b49700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 0x7faafc074060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b530 tx=0x7faaec00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec011830 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc1a2160 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaec011e70 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00f340 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.297+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc1a26e0 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.298+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc109060 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.299+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7faaec0106d0 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.300+0000 7faaf37fe700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faae80a20c0 0x7faae80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.300+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7faaec0cbf80 con 0x7faafc102ea0 2024-02-26T06:19:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.300+0000 7fab0234a700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faae80a20c0 0x7faae80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:19:38.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.301+0000 7fab0234a700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faae80a20c0 0x7faae80a4580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faafc102d00 tx=0x7faaf800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:19:38.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.304+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faaec0967a0 con 0x7faafc102ea0 2024-02-26T06:19:38.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:38.505+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7faafc02d870 con 0x7faae80a20c0 2024-02-26T06:19:39.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:38 smithi090 ceph-mon[35681]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T06:19:39.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:38 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:39.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:38 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:39.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:38 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:39.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:38 smithi167 ceph-mon[36229]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T06:19:39.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:38 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:39.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:38 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:39.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:38 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:39.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:38 smithi006 ceph-mon[31790]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T06:19:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:38 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:19:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:38 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:19:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:38 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:40.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:39 smithi090 ceph-mon[35681]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:40.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:39 smithi090 ceph-mon[35681]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:40.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:39 smithi167 ceph-mon[36229]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:40.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:39 smithi167 ceph-mon[36229]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:40.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:39 smithi006 ceph-mon[31790]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:40.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:39 smithi006 ceph-mon[31790]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:19:42.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:42.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/1007376528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]: dispatch 2024-02-26T06:19:42.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]: dispatch 2024-02-26T06:19:42.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]': finished 2024-02-26T06:19:42.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T06:19:42.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:19:42.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:41 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/2993837564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/1007376528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]: dispatch 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]: dispatch 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]': finished 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:19:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:41 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/2993837564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/1007376528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]: dispatch 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]: dispatch 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8b7b566-c20f-4c25-83c7-3cc834dcbaca"}]': finished 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:19:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:41 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/2993837564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:19:44.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:43 smithi006 ceph-mon[31790]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:44.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:43 smithi167 ceph-mon[36229]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:43 smithi090 ceph-mon[35681]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:46.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:45 smithi006 ceph-mon[31790]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:46.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:45 smithi167 ceph-mon[36229]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:45 smithi090 ceph-mon[35681]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:47.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:47 smithi167 ceph-mon[36229]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:48.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:47 smithi006 ceph-mon[31790]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:47 smithi090 ceph-mon[35681]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:49.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:48 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T06:19:49.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:48 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:49.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:48 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T06:19:49.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:48 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:49.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:48 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T06:19:49.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:48 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:19:50.164 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:49 smithi167 ceph-mon[36229]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:50.164 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:49 smithi167 ceph-mon[36229]: Deploying daemon osd.2 on smithi167 2024-02-26T06:19:50.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:49 smithi006 ceph-mon[31790]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:50.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:49 smithi006 ceph-mon[31790]: Deploying daemon osd.2 on smithi167 2024-02-26T06:19:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:49 smithi090 ceph-mon[35681]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:49 smithi090 ceph-mon[35681]: Deploying daemon osd.2 on smithi167 2024-02-26T06:19:52.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:51 smithi167 ceph-mon[36229]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:52.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:51 smithi006 ceph-mon[31790]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:52.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:51 smithi090 ceph-mon[35681]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:54.242 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:53 smithi167 ceph-mon[36229]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:54.242 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:54.242 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:54.242 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:53 smithi090 ceph-mon[35681]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:54.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:53 smithi006 ceph-mon[31790]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:54.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:54.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:54.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:19:56.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:55 smithi167 ceph-mon[36229]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:55 smithi090 ceph-mon[35681]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:56.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:55 smithi006 ceph-mon[31790]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:56.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.693+0000 7faaf37fe700 1 -- 172.21.15.167:0/2042871621 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faafc02d870 con 0x7faae80a20c0 2024-02-26T06:19:56.694 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-02-26T06:19:56.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faae80a20c0 msgr2=0x7faae80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faae80a20c0 0x7faae80a4580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faafc102d00 tx=0x7faaf800b410 comp rx=0 tx=0).stop 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 msgr2=0x7faafc074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 0x7faafc074060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b530 tx=0x7faaec00b840 comp rx=0 tx=0).stop 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 shutdown_connections 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faae80a20c0 0x7faae80a4580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faafc103840 0x7faafc0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc101ca0 0x7faafc075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 --2- 172.21.15.167:0/2042871621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc102ea0 0x7faafc074060 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.696+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 >> 172.21.15.167:0/2042871621 conn(0x7faafc0fd280 msgr2=0x7faafc1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.697+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 shutdown_connections 2024-02-26T06:19:56.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:19:56.697+0000 7fab045ae700 1 -- 172.21.15.167:0/2042871621 wait complete. 2024-02-26T06:19:57.613 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.2.service 2024-02-26T06:19:57.616 INFO:tasks.cephadm:Deploying osd.3 on smithi006 with /dev/vg_nvme/lv_3... 2024-02-26T06:19:57.616 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T06:19:57.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:57 smithi167 ceph-mon[36229]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:57.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:57.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:57.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:57 smithi167 ceph-mon[36229]: from='osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:19:57.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:57 smithi167 ceph-mon[36229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:19:58.077 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:19:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:57 smithi090 ceph-mon[35681]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:57 smithi090 ceph-mon[35681]: from='osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:19:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:57 smithi090 ceph-mon[35681]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:19:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:57 smithi006 ceph-mon[31790]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:19:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:57 smithi006 ceph-mon[31790]: from='osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:19:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:57 smithi006 ceph-mon[31790]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T06:19:58.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T06:19:58.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T06:19:58.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:19:58.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: from='osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:19:58.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:58 smithi006 ceph-mon[31790]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:19:58.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T06:19:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T06:19:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:19:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: from='osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:19:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:58 smithi167 ceph-mon[36229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:19:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:19:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T06:19:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T06:19:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:19:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: from='osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:19:59.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:58 smithi090 ceph-mon[35681]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:19:59.940 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2[41410]: 2024-02-26T06:19:59.786+0000 7f260922e700 -1 osd.2 0 waiting for initial osdmap 2024-02-26T06:19:59.940 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2[41410]: 2024-02-26T06:19:59.798+0000 7f260181a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:20:00.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-mon[36229]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:20:00.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-mon[36229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:20:00.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-mon[36229]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T06:20:00.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:00.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:00.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:19:59 smithi167 ceph-mon[36229]: from='osd.2 ' entity='osd.2' 2024-02-26T06:20:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:59 smithi090 ceph-mon[35681]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:20:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:59 smithi090 ceph-mon[35681]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:20:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:59 smithi090 ceph-mon[35681]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T06:20:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:19:59 smithi090 ceph-mon[35681]: from='osd.2 ' entity='osd.2' 2024-02-26T06:20:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:59 smithi006 ceph-mon[31790]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:20:00.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:59 smithi006 ceph-mon[31790]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:20:00.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:59 smithi006 ceph-mon[31790]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T06:20:00.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:00.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:00.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:19:59 smithi006 ceph-mon[31790]: from='osd.2 ' entity='osd.2' 2024-02-26T06:20:00.880 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T06:20:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T06:20:00.881 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-02-26T06:20:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-02-26T06:20:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0344802 s, 304 MB/s 2024-02-26T06:20:00.881 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 2024-02-26T06:20:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-02-26T06:20:01.168 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:20:01.168 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:20:01.168 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:01.168 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: overall HEALTH_OK 2024-02-26T06:20:01.169 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] boot 2024-02-26T06:20:01.169 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T06:20:01.169 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:20:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:20:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: overall HEALTH_OK 2024-02-26T06:20:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] boot 2024-02-26T06:20:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T06:20:01.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: overall HEALTH_OK 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: osd.2 [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] boot 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T06:20:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:20:01.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_3 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi167 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi167 to 17080M 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:02.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:02.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:02.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.160 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi167 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi167 to 17080M 2024-02-26T06:20:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:02.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi167 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi167 to 17080M 2024-02-26T06:20:02.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:02.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:03.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:02 smithi167 ceph-mon[36229]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:03.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:20:03.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T06:20:03.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:02 smithi167 ceph-mon[36229]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T06:20:03.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:20:03.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:02 smithi006 ceph-mon[31790]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:03.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:20:03.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T06:20:03.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:02 smithi006 ceph-mon[31790]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T06:20:03.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:20:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:02 smithi090 ceph-mon[35681]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:20:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T06:20:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:02 smithi090 ceph-mon[35681]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T06:20:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:20:04.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 -- 172.21.15.6:0/3179640800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380075ca0 msgr2=0x7fc380074100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 --2- 172.21.15.6:0/3179640800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380075ca0 0x7fc380074100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc374009990 tx=0x7fc374009ca0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 -- 172.21.15.6:0/3179640800 shutdown_connections 2024-02-26T06:20:04.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 --2- 172.21.15.6:0/3179640800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc380109b80 0x7fc38010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 --2- 172.21.15.6:0/3179640800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380075ca0 0x7fc380074100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 --2- 172.21.15.6:0/3179640800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3800746d0 0x7fc380074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 -- 172.21.15.6:0/3179640800 >> 172.21.15.6:0/3179640800 conn(0x7fc3800fd210 msgr2=0x7fc3800ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:04.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 -- 172.21.15.6:0/3179640800 shutdown_connections 2024-02-26T06:20:04.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.187+0000 7fc387a00700 1 -- 172.21.15.6:0/3179640800 wait complete. 2024-02-26T06:20:04.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.188+0000 7fc387a00700 1 Processor -- start 2024-02-26T06:20:04.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.188+0000 7fc387a00700 1 -- start start 2024-02-26T06:20:04.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc387a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3800746d0 0x7fc380104fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc387a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 0x7fc380105520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc387a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380109b80 0x7fc380101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc387a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380102300 con 0x7fc380075ca0 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc387a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380102470 con 0x7fc3800746d0 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc387a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380102630 con 0x7fc380109b80 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc37fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 0x7fc380105520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc37fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 0x7fc380105520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50868/0 (socket says 172.21.15.6:50868) 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc37fe00700 1 -- 172.21.15.6:0/3124038682 learned_addr learned my addr 172.21.15.6:0/3124038682 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc385000700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3800746d0 0x7fc380104fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.189+0000 7fc385a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380109b80 0x7fc380101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37fe00700 1 -- 172.21.15.6:0/3124038682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380109b80 msgr2=0x7fc380101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37fe00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380109b80 0x7fc380101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37fe00700 1 -- 172.21.15.6:0/3124038682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3800746d0 msgr2=0x7fc380104fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37fe00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3800746d0 0x7fc380104fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37fe00700 1 -- 172.21.15.6:0/3124038682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc378009c40 con 0x7fc380075ca0 2024-02-26T06:20:04.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37fe00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 0x7fc380105520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00e970 tx=0x7fc36c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:04.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c009790 con 0x7fc380075ca0 2024-02-26T06:20:04.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.190+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3740095f0 con 0x7fc380075ca0 2024-02-26T06:20:04.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.191+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc36c00cca0 con 0x7fc380075ca0 2024-02-26T06:20:04.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.191+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c003cf0 con 0x7fc380075ca0 2024-02-26T06:20:04.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.191+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3801a67b0 con 0x7fc380075ca0 2024-02-26T06:20:04.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.192+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc380108940 con 0x7fc380075ca0 2024-02-26T06:20:04.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.194+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7fc36c00c7c0 con 0x7fc380075ca0 2024-02-26T06:20:04.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.196+0000 7fc37d600700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3680a20c0 0x7fc3680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:04.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.196+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fc36c014070 con 0x7fc380075ca0 2024-02-26T06:20:04.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.196+0000 7fc385000700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3680a20c0 0x7fc3680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:04.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.197+0000 7fc385000700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3680a20c0 0x7fc3680a4580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc37400b650 tx=0x7fc374005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:04.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.199+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc36c096f80 con 0x7fc380075ca0 2024-02-26T06:20:04.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.433+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc38002d800 con 0x7fc3680a20c0 2024-02-26T06:20:04.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:04.694+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc36c097350 con 0x7fc380075ca0 2024-02-26T06:20:04.695 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:04 smithi006 sudo[51012]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:20:04.695 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:04 smithi006 sudo[51012]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:04.695 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:04 smithi006 sudo[51012]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:04.695 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:04 smithi006 sudo[51012]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:04.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:04 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:20:04.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:04 smithi006 ceph-mon[31790]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T06:20:04.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:04 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:20:04.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:04 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:20:04.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:04 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:04.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:04 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:20:04.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:04 smithi167 ceph-mon[36229]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T06:20:04.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:04 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:20:04.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:04 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:20:04.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:04 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:04 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:20:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:04 smithi090 ceph-mon[35681]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T06:20:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:04 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:20:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:04 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:20:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:04 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:05.366 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:05 smithi006 sudo[51082]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:20:05.367 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:05 smithi006 sudo[51082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:05.367 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:05 smithi006 sudo[51082]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:05.367 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:20:05 smithi006 sudo[51082]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:05.403 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44619]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:20:05.403 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:05.403 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44619]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:05.403 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44619]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:05.680 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44630]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:20:05.680 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:05.680 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44630]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:05.680 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:20:05 smithi090 sudo[44630]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:05.994 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:05 smithi167 sudo[45362]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:20:05.994 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:05 smithi167 sudo[45362]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:05.994 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:05 smithi167 sudo[45362]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:05.995 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:05 smithi167 sudo[45362]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:05.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:05 smithi167 ceph-mon[36229]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:05.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:05 smithi167 ceph-mon[36229]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:05.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:05 smithi167 ceph-mon[36229]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T06:20:05.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:05 smithi167 ceph-mon[36229]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T06:20:05.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:05.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:05 smithi090 ceph-mon[35681]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:05 smithi090 ceph-mon[35681]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:05 smithi090 ceph-mon[35681]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T06:20:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:05 smithi090 ceph-mon[35681]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T06:20:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:05 smithi006 ceph-mon[31790]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:05 smithi006 ceph-mon[31790]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:05 smithi006 ceph-mon[31790]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T06:20:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:05 smithi006 ceph-mon[31790]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T06:20:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:06.494 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:06 smithi167 sudo[45365]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:20:06.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:06 smithi167 sudo[45365]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:06.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:06 smithi167 sudo[45365]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:06.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:20:06 smithi167 sudo[45365]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:06.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:06 smithi006 sudo[51162]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:20:06.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:06 smithi006 sudo[51162]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:06.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:06 smithi006 sudo[51162]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:06.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:06 smithi006 sudo[51162]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:07.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:07 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:07.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:07 smithi090 ceph-mon[35681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 ceph-mon[36229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45368]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:20:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45368]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45368]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45368]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:07.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:07 smithi006 sudo[51190]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:20:07.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:07 smithi006 sudo[51190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:07 smithi006 sudo[51190]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:07 smithi006 sudo[51190]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:07 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:07 smithi006 ceph-mon[31790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:07.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45371]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:20:07.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:07.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45371]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:07.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:07 smithi167 sudo[45371]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:08.346 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44636]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T06:20:08.346 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:08.346 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44636]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:08.346 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44636]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44639]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44639]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 sudo[44639]: pam_unix(sudo:session): session closed for user root 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:08 smithi090 ceph-mon[35681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:08 smithi006 ceph-mon[31790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T06:20:08.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:20:08.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:20:08.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:20:08.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:08 smithi167 ceph-mon[36229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T06:20:09.573 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:09 smithi006 ceph-mon[31790]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:09.574 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:09 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2486067403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]: dispatch 2024-02-26T06:20:09.574 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:09 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]: dispatch 2024-02-26T06:20:09.574 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:09 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]': finished 2024-02-26T06:20:09.574 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:09 smithi006 ceph-mon[31790]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T06:20:09.574 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:09.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:09 smithi167 ceph-mon[36229]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:09.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:09 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2486067403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]: dispatch 2024-02-26T06:20:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:09 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]: dispatch 2024-02-26T06:20:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:09 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]': finished 2024-02-26T06:20:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:09 smithi167 ceph-mon[36229]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T06:20:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:09 smithi090 ceph-mon[35681]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:09 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2486067403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]: dispatch 2024-02-26T06:20:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:09 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]: dispatch 2024-02-26T06:20:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:09 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513bca2a-d4e4-4934-8826-3b204caa92b6"}]': finished 2024-02-26T06:20:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:09 smithi090 ceph-mon[35681]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T06:20:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:10.675 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:10 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3492358870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:20:10.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:10 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3492358870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:20:10.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:10 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3492358870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:20:11.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:11 smithi006 ceph-mon[31790]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:11 smithi167 ceph-mon[36229]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:11 smithi090 ceph-mon[35681]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:13.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:13 smithi006 ceph-mon[31790]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:13.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:13 smithi167 ceph-mon[36229]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:13.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:13 smithi090 ceph-mon[35681]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:15.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:15 smithi006 ceph-mon[31790]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-26T06:20:15.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:15 smithi167 ceph-mon[36229]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-26T06:20:15.849 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:15 smithi090 ceph-mon[35681]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-26T06:20:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:17 smithi006 ceph-mon[31790]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-26T06:20:17.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:17 smithi167 ceph-mon[36229]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-26T06:20:17.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:17 smithi090 ceph-mon[35681]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-26T06:20:19.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:19 smithi006 ceph-mon[31790]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-26T06:20:19.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:19 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T06:20:19.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:19 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:19.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:19 smithi006 ceph-mon[31790]: Deploying daemon osd.3 on smithi006 2024-02-26T06:20:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:19 smithi167 ceph-mon[36229]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-26T06:20:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:19 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T06:20:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:19 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:19 smithi167 ceph-mon[36229]: Deploying daemon osd.3 on smithi006 2024-02-26T06:20:19.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:19 smithi090 ceph-mon[35681]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-26T06:20:19.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:19 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T06:20:19.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:19 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:19.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:19 smithi090 ceph-mon[35681]: Deploying daemon osd.3 on smithi006 2024-02-26T06:20:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:21 smithi006 ceph-mon[31790]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 898 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-26T06:20:21.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:21 smithi167 ceph-mon[36229]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 898 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-26T06:20:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:21 smithi090 ceph-mon[35681]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 898 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-26T06:20:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:23 smithi167 ceph-mon[36229]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:20:23.810 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:23 smithi006 ceph-mon[31790]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:20:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:23 smithi090 ceph-mon[35681]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:20:25.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:25 smithi006 ceph-mon[31790]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:20:25.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:25 smithi167 ceph-mon[36229]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:20:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:25 smithi090 ceph-mon[35681]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-26T06:20:27.821 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:27 smithi006 ceph-mon[31790]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:27.821 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:27.821 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:27.821 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:27.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:27 smithi090 ceph-mon[35681]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:27.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:27.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:27.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:27.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:27 smithi167 ceph-mon[36229]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:27.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:27.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:29.680 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:29 smithi006 ceph-mon[31790]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:29 smithi090 ceph-mon[35681]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:29.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:29 smithi167 ceph-mon[36229]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:31.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:20:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:30 smithi167 ceph-mon[36229]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:20:31.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:30 smithi090 ceph-mon[35681]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:31.367 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:20:31.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:30 smithi006 ceph-mon[31790]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T06:20:31.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.472+0000 7fc37d600700 1 -- 172.21.15.6:0/3124038682 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc38002d800 con 0x7fc3680a20c0 2024-02-26T06:20:31.473 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 3 on host 'smithi006' 2024-02-26T06:20:31.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.475+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3680a20c0 msgr2=0x7fc3680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.475+0000 7fc387a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3680a20c0 0x7fc3680a4580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc37400b650 tx=0x7fc374005fb0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.475+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 msgr2=0x7fc380105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:31.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.475+0000 7fc387a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 0x7fc380105520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00e970 tx=0x7fc36c00ec80 comp rx=0 tx=0).stop 2024-02-26T06:20:31.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 shutdown_connections 2024-02-26T06:20:31.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3680a20c0 0x7fc3680a4580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc380109b80 0x7fc380101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3800746d0 0x7fc380104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 --2- 172.21.15.6:0/3124038682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc380075ca0 0x7fc380105520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:31.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 >> 172.21.15.6:0/3124038682 conn(0x7fc3800fd210 msgr2=0x7fc380105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:31.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 shutdown_connections 2024-02-26T06:20:31.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:20:31.476+0000 7fc387a00700 1 -- 172.21.15.6:0/3124038682 wait complete. 2024-02-26T06:20:32.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T06:20:32.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:20:32.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T06:20:32.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:32.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:20:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:32 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:32.231 DEBUG:teuthology.orchestra.run.smithi006:osd.3> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.3.service 2024-02-26T06:20:32.237 INFO:tasks.cephadm:Deploying osd.4 on smithi090 with /dev/vg_nvme/lv_3... 2024-02-26T06:20:32.237 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T06:20:32.267 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:32.268 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:32 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:32.481 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:20:32.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:32.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:32 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:33 smithi090 ceph-mon[35681]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:33 smithi090 ceph-mon[35681]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:20:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:33 smithi090 ceph-mon[35681]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T06:20:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:33 smithi006 ceph-mon[31790]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:33 smithi006 ceph-mon[31790]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:20:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:33 smithi006 ceph-mon[31790]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T06:20:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:33 smithi167 ceph-mon[36229]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-02-26T06:20:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:33 smithi167 ceph-mon[36229]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:20:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:33 smithi167 ceph-mon[36229]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T06:20:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:34.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:34 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:20:34.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:34 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:20:34.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:34 smithi090 ceph-mon[35681]: osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] boot 2024-02-26T06:20:34.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:34 smithi090 ceph-mon[35681]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T06:20:34.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:34 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:34.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:34 smithi090 ceph-mon[35681]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T06:20:34.333 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T06:20:34.333 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T06:20:34.333 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-02-26T06:20:34.333 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-02-26T06:20:34.334 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116712 s, 898 MB/s 2024-02-26T06:20:34.334 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-02-26T06:20:34.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:34 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:20:34.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:34 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:20:34.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:34 smithi006 ceph-mon[31790]: osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] boot 2024-02-26T06:20:34.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:34 smithi006 ceph-mon[31790]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T06:20:34.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:34 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:34.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:34 smithi006 ceph-mon[31790]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T06:20:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:34 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:20:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:34 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:20:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:34 smithi167 ceph-mon[36229]: osd.3 [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] boot 2024-02-26T06:20:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:34 smithi167 ceph-mon[36229]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T06:20:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:34 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:20:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:34 smithi167 ceph-mon[36229]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T06:20:34.920 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-02-26T06:20:35.202 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:20:35.283 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:35 smithi006 ceph-mon[31790]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:35.283 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:35 smithi006 ceph-mon[31790]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T06:20:35.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:35 smithi090 ceph-mon[35681]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:35.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:35 smithi090 ceph-mon[35681]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T06:20:35.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:35 smithi167 ceph-mon[36229]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:35.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:35 smithi167 ceph-mon[36229]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 -- 172.21.15.90:0/2949907868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064074650 msgr2=0x7fe064074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 --2- 172.21.15.90:0/2949907868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064074650 0x7fe064074ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009990 tx=0x7fe04c009ca0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 -- 172.21.15.90:0/2949907868 shutdown_connections 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 --2- 172.21.15.90:0/2949907868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064109ba0 0x7fe06410c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 --2- 172.21.15.90:0/2949907868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064075c20 0x7fe064074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 --2- 172.21.15.90:0/2949907868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064074650 0x7fe064074ad0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 -- 172.21.15.90:0/2949907868 >> 172.21.15.90:0/2949907868 conn(0x7fe0640fd280 msgr2=0x7fe0640ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 -- 172.21.15.90:0/2949907868 shutdown_connections 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 -- 172.21.15.90:0/2949907868 wait complete. 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 Processor -- start 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.793+0000 7fe069dee700 1 -- start start 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064074650 0x7fe064199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 0x7fe06419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064109ba0 0x7fe0641a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0641a19f0 con 0x7fe064109ba0 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0641a1b30 con 0x7fe064075c20 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0641a1cc0 con 0x7fe064074650 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 0x7fe06419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 0x7fe06419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45150/0 (socket says 172.21.15.90:45150) 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 -- 172.21.15.90:0/3724048038 learned_addr learned my addr 172.21.15.90:0/3724048038 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe0637fe700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064074650 0x7fe064199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe063fff700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064109ba0 0x7fe0641a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 -- 172.21.15.90:0/3724048038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064074650 msgr2=0x7fe064199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064074650 0x7fe064199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 -- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064109ba0 msgr2=0x7fe0641a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064109ba0 0x7fe0641a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 -- 172.21.15.90:0/3724048038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0095f0 con 0x7fe064075c20 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe0637fe700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064074650 0x7fe064199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe063fff700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064109ba0 0x7fe0641a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe05bfff700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 0x7fe06419a3a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00b5c0 tx=0x7fe04c00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c01d070 con 0x7fe064075c20 2024-02-26T06:20:36.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0641a1f40 con 0x7fe064075c20 2024-02-26T06:20:36.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe04c00fc30 con 0x7fe064075c20 2024-02-26T06:20:36.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c017650 con 0x7fe064075c20 2024-02-26T06:20:36.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.794+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0641a22f0 con 0x7fe064075c20 2024-02-26T06:20:36.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.796+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0641089b0 con 0x7fe064075c20 2024-02-26T06:20:36.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.796+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe04c0177b0 con 0x7fe064075c20 2024-02-26T06:20:36.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.797+0000 7fe0617fa700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe0500a2150 0x7fe0500a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:20:36.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.797+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3669+0+0 (secure 0 0 0) 0x7fe04c0cdf40 con 0x7fe064075c20 2024-02-26T06:20:36.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.797+0000 7fe0637fe700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe0500a2150 0x7fe0500a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:20:36.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.798+0000 7fe0637fe700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe0500a2150 0x7fe0500a4610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0540099d0 tx=0x7fe054008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:20:36.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:36.801+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe04c0981c0 con 0x7fe064075c20 2024-02-26T06:20:37.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:37.007+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe06402d950 con 0x7fe0500a2150 2024-02-26T06:20:37.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:37 smithi167 ceph-mon[36229]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:37.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:37 smithi167 ceph-mon[36229]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T06:20:37.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:20:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:20:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:37.344 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:37 smithi006 ceph-mon[31790]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:37.345 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:37 smithi006 ceph-mon[31790]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T06:20:37.345 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:20:37.345 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:20:37.345 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:37 smithi090 ceph-mon[35681]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:37 smithi090 ceph-mon[35681]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T06:20:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:20:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:20:37.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:38.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:38 smithi006 ceph-mon[31790]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:38.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:38 smithi006 ceph-mon[31790]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T06:20:38.264 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:38 smithi090 ceph-mon[35681]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:38.264 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:38 smithi090 ceph-mon[35681]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T06:20:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:38 smithi167 ceph-mon[36229]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:20:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:38 smithi167 ceph-mon[36229]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T06:20:39.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi006 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi006 to 8547M 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:39.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:39.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:39 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi006 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi006 to 8547M 2024-02-26T06:20:39.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:39.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:39.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:39 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:20:39.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi006 2024-02-26T06:20:39.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:39.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:20:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi006 to 8547M 2024-02-26T06:20:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:20:39.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:39 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:40 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/1279083135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]: dispatch 2024-02-26T06:20:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:40 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]: dispatch 2024-02-26T06:20:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:40 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]': finished 2024-02-26T06:20:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:40 smithi090 ceph-mon[35681]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T06:20:40.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:40 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:40.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:40 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/1279083135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]: dispatch 2024-02-26T06:20:40.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:40 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]: dispatch 2024-02-26T06:20:40.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:40 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]': finished 2024-02-26T06:20:40.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:40 smithi167 ceph-mon[36229]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T06:20:40.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:40 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:40.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:40 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/1279083135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]: dispatch 2024-02-26T06:20:40.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:40 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]: dispatch 2024-02-26T06:20:40.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:40 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2ca8afc-575f-4cdd-8310-2c62fba98c84"}]': finished 2024-02-26T06:20:40.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:40 smithi006 ceph-mon[31790]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T06:20:40.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:40 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:41.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:41 smithi167 ceph-mon[36229]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-02-26T06:20:41.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:41 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/3468704922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:20:41.685 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:41 smithi006 ceph-mon[31790]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-02-26T06:20:41.685 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:41 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/3468704922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:20:41.685 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:41 smithi090 ceph-mon[35681]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-02-26T06:20:41.685 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:41 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/3468704922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:20:43.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:43 smithi167 ceph-mon[36229]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:20:43.549 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:43 smithi090 ceph-mon[35681]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:20:43.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:43 smithi006 ceph-mon[31790]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T06:20:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:45 smithi090 ceph-mon[35681]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:20:45.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:45 smithi006 ceph-mon[31790]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:20:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:45 smithi167 ceph-mon[36229]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:20:47.555 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:47 smithi090 ceph-mon[35681]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:47.555 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T06:20:47.555 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:47.555 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:47 smithi090 ceph-mon[35681]: Deploying daemon osd.4 on smithi090 2024-02-26T06:20:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:47 smithi006 ceph-mon[31790]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T06:20:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:47 smithi006 ceph-mon[31790]: Deploying daemon osd.4 on smithi090 2024-02-26T06:20:47.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:47 smithi167 ceph-mon[36229]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T06:20:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:20:47.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:47 smithi167 ceph-mon[36229]: Deploying daemon osd.4 on smithi090 2024-02-26T06:20:49.579 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:49 smithi090 ceph-mon[35681]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:49.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:49 smithi006 ceph-mon[31790]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:49.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:49 smithi167 ceph-mon[36229]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:51.355 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:51 smithi090 ceph-mon[35681]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:51.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:51 smithi006 ceph-mon[31790]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:51 smithi167 ceph-mon[36229]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:53.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:53 smithi167 ceph-mon[36229]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:53.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:53.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:53.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:53.603 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:53 smithi090 ceph-mon[35681]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:53.603 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:53.603 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:53.603 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:53.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:53 smithi006 ceph-mon[31790]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:53.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:53.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:20:53.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:55 smithi167 ceph-mon[36229]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:55.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:55 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:55 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.533 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:55 smithi090 ceph-mon[35681]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:55.533 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:55 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.533 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:55 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:55 smithi006 ceph-mon[31790]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:55.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:55 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:55 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:55.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.884+0000 7fe0617fa700 1 -- 172.21.15.90:0/3724048038 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe06402d950 con 0x7fe0500a2150 2024-02-26T06:20:55.884 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 4 on host 'smithi090' 2024-02-26T06:20:55.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe0500a2150 msgr2=0x7fe0500a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:55.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe0500a2150 0x7fe0500a4610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0540099d0 tx=0x7fe054008040 comp rx=0 tx=0).stop 2024-02-26T06:20:55.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 msgr2=0x7fe06419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:20:55.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 0x7fe06419a3a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00b5c0 tx=0x7fe04c00f740 comp rx=0 tx=0).stop 2024-02-26T06:20:55.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 shutdown_connections 2024-02-26T06:20:55.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe0500a2150 0x7fe0500a4610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:55.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe064109ba0 0x7fe0641a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:55.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe064074650 0x7fe064199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:55.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 --2- 172.21.15.90:0/3724048038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe064075c20 0x7fe06419a3a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:20:55.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 >> 172.21.15.90:0/3724048038 conn(0x7fe0640fd280 msgr2=0x7fe064105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:20:55.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 shutdown_connections 2024-02-26T06:20:55.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:20:55.886+0000 7fe069dee700 1 -- 172.21.15.90:0/3724048038 wait complete. 2024-02-26T06:20:56.468 DEBUG:teuthology.orchestra.run.smithi090:osd.4> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.4.service 2024-02-26T06:20:56.471 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-02-26T06:20:56.471 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T06:20:56.706 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:20:56.976 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:56 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:56.976 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:56 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:56.976 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:56 smithi090 ceph-mon[35681]: from='osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:20:56.976 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:56 smithi090 ceph-mon[35681]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:20:57.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:56 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:57.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:56 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:57.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:56 smithi167 ceph-mon[36229]: from='osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:20:57.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:56 smithi167 ceph-mon[36229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:20:57.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:56 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:56 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:20:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:56 smithi006 ceph-mon[31790]: from='osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:20:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:56 smithi006 ceph-mon[31790]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T06:20:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:57 smithi090 ceph-mon[35681]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:57 smithi090 ceph-mon[35681]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T06:20:58.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:57 smithi090 ceph-mon[35681]: from='osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:20:58.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:57 smithi090 ceph-mon[35681]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T06:20:58.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:58.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:57 smithi090 ceph-mon[35681]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:20:58.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:57 smithi167 ceph-mon[36229]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:58.131 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:57 smithi167 ceph-mon[36229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T06:20:58.131 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:57 smithi167 ceph-mon[36229]: from='osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:20:58.131 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:57 smithi167 ceph-mon[36229]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T06:20:58.131 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:58.131 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:57 smithi167 ceph-mon[36229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:20:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:57 smithi006 ceph-mon[31790]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:20:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:57 smithi006 ceph-mon[31790]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T06:20:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:57 smithi006 ceph-mon[31790]: from='osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:20:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:57 smithi006 ceph-mon[31790]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T06:20:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:57 smithi006 ceph-mon[31790]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:20:59.007 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T06:20:59.007 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T06:20:59.007 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-02-26T06:20:59.007 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-02-26T06:20:59.007 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120403 s, 871 MB/s 2024-02-26T06:20:59.007 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-02-26T06:20:59.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:58 smithi167 ceph-mon[36229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:20:59.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:58 smithi167 ceph-mon[36229]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T06:20:59.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:59.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:58 smithi167 ceph-mon[36229]: from='osd.4 ' entity='osd.4' 2024-02-26T06:20:59.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-mon[35681]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:20:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-mon[35681]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T06:20:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-mon[35681]: from='osd.4 ' entity='osd.4' 2024-02-26T06:20:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:59.116 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:20:58.745+0000 7f92be43f700 -1 osd.4 0 waiting for initial osdmap 2024-02-26T06:20:59.116 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:20:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:20:58.754+0000 7f92b6a2b700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:20:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:58 smithi006 ceph-mon[31790]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:20:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:58 smithi006 ceph-mon[31790]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T06:20:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:58 smithi006 ceph-mon[31790]: from='osd.4 ' entity='osd.4' 2024-02-26T06:20:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:20:59.573 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-02-26T06:20:59.810 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:00.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] boot 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T06:21:00.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:20:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] boot 2024-02-26T06:21:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T06:21:00.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:20:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:00.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:00.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:00.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: osd.4 [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] boot 2024-02-26T06:21:00.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T06:21:00.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:20:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:21:01.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:00 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi090 2024-02-26T06:21:01.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:00 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi090 to 8540M 2024-02-26T06:21:01.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:01.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:01.150 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:01.150 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:00 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi090 2024-02-26T06:21:01.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:00 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi090 to 8540M 2024-02-26T06:21:01.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:01.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:01.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:01.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 -- 172.21.15.167:0/1544122064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d01050d0 msgr2=0x7f07d01074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/1544122064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d01050d0 0x7f07d01074c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009990 tx=0x7f07c4009ca0 comp rx=0 tx=0).stop 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 -- 172.21.15.167:0/1544122064 shutdown_connections 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/1544122064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d0107a00 0x7f07d0109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/1544122064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01027a0 0x7f07d0104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/1544122064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d01050d0 0x7f07d01074c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 -- 172.21.15.167:0/1544122064 >> 172.21.15.167:0/1544122064 conn(0x7f07d00fc1d0 msgr2=0x7f07d00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 -- 172.21.15.167:0/1544122064 shutdown_connections 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 -- 172.21.15.167:0/1544122064 wait complete. 2024-02-26T06:21:01.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 Processor -- start 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.364+0000 7f07d6eb4700 1 -- start start 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d01027a0 0x7f07d0102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01050d0 0x7f07d0100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 0x7f07d0100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01013a0 con 0x7f07d01027a0 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0101510 con 0x7f07d01050d0 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01a1da0 con 0x7f07d0107a00 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 0x7f07d0100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 0x7f07d0100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37958/0 (socket says 172.21.15.167:37958) 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 -- 172.21.15.167:0/2503899528 learned_addr learned my addr 172.21.15.167:0/2503899528 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07cffff700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01050d0 0x7f07d0100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 -- 172.21.15.167:0/2503899528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01050d0 msgr2=0x7f07d0100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01050d0 0x7f07d0100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 -- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d01027a0 msgr2=0x7f07d0102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:01.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d01027a0 0x7f07d0102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 -- 172.21.15.167:0/2503899528 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c40095f0 con 0x7f07d0107a00 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07cffff700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01050d0 0x7f07d0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d5451700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 0x7f07d0100c90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07c800b9e0 tx=0x7f07c800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800c9b0 con 0x7f07d0107a00 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07c8014440 con 0x7f07d0107a00 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d01a1fa0 con 0x7f07d0107a00 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.365+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80125a0 con 0x7f07d0107a00 2024-02-26T06:21:01.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.366+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d01a24a0 con 0x7f07d0107a00 2024-02-26T06:21:01.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:00 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi090 2024-02-26T06:21:01.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:00 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi090 to 8540M 2024-02-26T06:21:01.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:01.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:01.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:01.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:01.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.367+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d00fdd50 con 0x7f07d0107a00 2024-02-26T06:21:01.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.367+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f07c8014960 con 0x7f07d0107a00 2024-02-26T06:21:01.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.368+0000 7f07cdffb700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07b80a2150 0x7f07b80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:01.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.368+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f07c8006e40 con 0x7f07d0107a00 2024-02-26T06:21:01.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.368+0000 7f07d4c50700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07b80a2150 0x7f07b80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:01.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.369+0000 7f07d4c50700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07b80a2150 0x7f07b80a4610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009770 tx=0x7f07c0006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:01.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.373+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07c80962e0 con 0x7f07d0107a00 2024-02-26T06:21:01.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:01.576+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f07d002d950 con 0x7f07b80a2150 2024-02-26T06:21:02.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:01 smithi167 ceph-mon[36229]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:02.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:01 smithi167 ceph-mon[36229]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T06:21:02.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:21:02.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:21:02.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:01 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:01 smithi090 ceph-mon[35681]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:01 smithi090 ceph-mon[35681]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T06:21:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:21:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:21:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:01 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:01 smithi006 ceph-mon[31790]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:01 smithi006 ceph-mon[31790]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T06:21:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:21:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:21:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:01 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:03.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:02 smithi167 ceph-mon[36229]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:21:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:02 smithi090 ceph-mon[35681]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:21:03.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:02 smithi006 ceph-mon[31790]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:21:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:03 smithi167 ceph-mon[36229]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:03 smithi090 ceph-mon[35681]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:04.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:03 smithi006 ceph-mon[31790]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:05 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/1879220888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]: dispatch 2024-02-26T06:21:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:05 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]: dispatch 2024-02-26T06:21:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:05 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]': finished 2024-02-26T06:21:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:05 smithi090 ceph-mon[35681]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T06:21:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:05 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/3811450806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:21:05.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:05 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/1879220888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]: dispatch 2024-02-26T06:21:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:05 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]: dispatch 2024-02-26T06:21:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:05 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]': finished 2024-02-26T06:21:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:05 smithi006 ceph-mon[31790]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T06:21:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:05.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:05 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/3811450806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:21:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:05 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/1879220888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]: dispatch 2024-02-26T06:21:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:05 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]: dispatch 2024-02-26T06:21:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:05 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5e79bb3-2052-4a0d-8b43-d6bf454109b8"}]': finished 2024-02-26T06:21:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:05 smithi167 ceph-mon[36229]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T06:21:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:05 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/3811450806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:21:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:06 smithi090 ceph-mon[35681]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:06 smithi006 ceph-mon[31790]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:06 smithi167 ceph-mon[36229]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:07 smithi090 ceph-mon[35681]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:07.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:07 smithi006 ceph-mon[31790]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:07.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:07 smithi167 ceph-mon[36229]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:09.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:09 smithi090 ceph-mon[35681]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:09.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:09 smithi006 ceph-mon[31790]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:09.422 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:09 smithi167 ceph-mon[36229]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:11.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:11 smithi090 ceph-mon[35681]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:11.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:11 smithi006 ceph-mon[31790]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:11.442 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:11 smithi167 ceph-mon[36229]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:12.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T06:21:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:12 smithi006 ceph-mon[31790]: Deploying daemon osd.5 on smithi167 2024-02-26T06:21:12.352 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T06:21:12.352 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:12.352 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:12 smithi167 ceph-mon[36229]: Deploying daemon osd.5 on smithi167 2024-02-26T06:21:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T06:21:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:12 smithi090 ceph-mon[35681]: Deploying daemon osd.5 on smithi167 2024-02-26T06:21:13.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:13 smithi167 ceph-mon[36229]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:13.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:13 smithi090 ceph-mon[35681]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:13.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:13 smithi006 ceph-mon[31790]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:15.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:15 smithi167 ceph-mon[36229]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:15.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:15 smithi090 ceph-mon[35681]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:15.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:15 smithi006 ceph-mon[31790]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:17 smithi090 ceph-mon[35681]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:17.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:17 smithi167 ceph-mon[36229]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:17.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:17.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:17.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:17 smithi006 ceph-mon[31790]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:19.351 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:19 smithi167 ceph-mon[36229]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:19.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:19 smithi090 ceph-mon[35681]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:19.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:19 smithi006 ceph-mon[31790]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:20.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.413+0000 7f07cdffb700 1 -- 172.21.15.167:0/2503899528 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f07d002d950 con 0x7f07b80a2150 2024-02-26T06:21:20.413 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-02-26T06:21:20.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07b80a2150 msgr2=0x7f07b80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:20.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07b80a2150 0x7f07b80a4610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009770 tx=0x7f07c0006cd0 comp rx=0 tx=0).stop 2024-02-26T06:21:20.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 msgr2=0x7f07d0100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 0x7f07d0100c90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07c800b9e0 tx=0x7f07c800bda0 comp rx=0 tx=0).stop 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 shutdown_connections 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07b80a2150 0x7f07b80a4610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07d0107a00 0x7f07d0100c90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d01027a0 0x7f07d0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 --2- 172.21.15.167:0/2503899528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07d01050d0 0x7f07d0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 >> 172.21.15.167:0/2503899528 conn(0x7f07d00fc1d0 msgr2=0x7f07d0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 shutdown_connections 2024-02-26T06:21:20.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:21:20.415+0000 7f07d6eb4700 1 -- 172.21.15.167:0/2503899528 wait complete. 2024-02-26T06:21:21.025 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.5.service 2024-02-26T06:21:21.028 INFO:tasks.cephadm:Deploying osd.6 on smithi006 with /dev/vg_nvme/lv_2... 2024-02-26T06:21:21.028 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T06:21:21.504 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:21.653 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:21.653 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.653 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.653 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: from='osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:21:21.654 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:21:21.654 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.654 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:21.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: from='osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:21:21.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:21:21.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: from='osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:22.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:22 smithi006 ceph-mon[31790]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T06:21:22.488 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:22 smithi006 ceph-mon[31790]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T06:21:22.488 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:22 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:22.488 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:22 smithi006 ceph-mon[31790]: from='osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:21:22.488 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:22 smithi006 ceph-mon[31790]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:21:22.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-mon[36229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T06:21:22.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-mon[36229]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T06:21:22.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:22.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-mon[36229]: from='osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:21:22.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-mon[36229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:21:22.745 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:21:22.415+0000 7f3f84c78700 -1 osd.5 0 waiting for initial osdmap 2024-02-26T06:21:22.745 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:21:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:21:22.422+0000 7f3f8126c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:21:22.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:22 smithi090 ceph-mon[35681]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T06:21:22.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:22 smithi090 ceph-mon[35681]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T06:21:22.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:22 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:22.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:22 smithi090 ceph-mon[35681]: from='osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:21:22.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:22 smithi090 ceph-mon[35681]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:21:23.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:23 smithi006 ceph-mon[31790]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:23.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:23 smithi006 ceph-mon[31790]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:21:23.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:23 smithi006 ceph-mon[31790]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T06:21:23.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:23.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:23.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:23 smithi167 ceph-mon[36229]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:23.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:23 smithi167 ceph-mon[36229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:21:23.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:23 smithi167 ceph-mon[36229]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T06:21:23.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:23.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:23 smithi090 ceph-mon[35681]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:23 smithi090 ceph-mon[35681]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:21:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:23 smithi090 ceph-mon[35681]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T06:21:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:23 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:23 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.480 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T06:21:24.480 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T06:21:24.480 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-02-26T06:21:24.480 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-02-26T06:21:24.481 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0339196 s, 309 MB/s 2024-02-26T06:21:24.481 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] boot 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: osdmap e38: 6 total, 6 up, 6 in 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:24.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:24.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:24.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:24.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:21:24.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:21:24.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] boot 2024-02-26T06:21:24.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: osdmap e38: 6 total, 6 up, 6 in 2024-02-26T06:21:24.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:24.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:24.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:24.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:24.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:21:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:21:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: osd.5 [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] boot 2024-02-26T06:21:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: osdmap e38: 6 total, 6 up, 6 in 2024-02-26T06:21:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:21:24.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:24.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:21:24.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:24.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:24.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:24.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:25.451 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_2 2024-02-26T06:21:25.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi167 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi167 to 8540M 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi167 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi167 to 8540M 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi167 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi167 to 8540M 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:25.911 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:21:26.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:26 smithi006 ceph-mon[31790]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T06:21:26.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:26 smithi167 ceph-mon[36229]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T06:21:26.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:26 smithi090 ceph-mon[35681]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T06:21:27.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:27 smithi167 ceph-mon[36229]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:27.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:27 smithi167 ceph-mon[36229]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T06:21:27.779 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:27 smithi006 ceph-mon[31790]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:27.779 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:27 smithi006 ceph-mon[31790]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T06:21:27.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:27 smithi090 ceph-mon[35681]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:27.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:27 smithi090 ceph-mon[35681]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 -- 172.21.15.6:0/3953232910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 msgr2=0x7ff7e410bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 --2- 172.21.15.6:0/3953232910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e410bf20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009f90 tx=0x7ff7dc00b330 comp rx=0 tx=0).stop 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 -- 172.21.15.6:0/3953232910 shutdown_connections 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 --2- 172.21.15.6:0/3953232910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e410bf20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 --2- 172.21.15.6:0/3953232910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e4075ca0 0x7ff7e4074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 --2- 172.21.15.6:0/3953232910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40746d0 0x7ff7e4074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.954+0000 7ff7e9200700 1 -- 172.21.15.6:0/3953232910 >> 172.21.15.6:0/3953232910 conn(0x7ff7e40fd180 msgr2=0x7ff7e40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.955+0000 7ff7e9200700 1 -- 172.21.15.6:0/3953232910 shutdown_connections 2024-02-26T06:21:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.955+0000 7ff7e9200700 1 -- 172.21.15.6:0/3953232910 wait complete. 2024-02-26T06:21:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 Processor -- start 2024-02-26T06:21:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 -- start start 2024-02-26T06:21:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40746d0 0x7ff7e4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:27.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e4075ca0 0x7ff7e419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:27.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e41a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:27.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e41a1a10 con 0x7ff7e40746d0 2024-02-26T06:21:27.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e41a1b50 con 0x7ff7e4109a90 2024-02-26T06:21:27.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.956+0000 7ff7e9200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e41a1ce0 con 0x7ff7e4075ca0 2024-02-26T06:21:27.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e41a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:27.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40746d0 0x7ff7e4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:27.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e41a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:45524/0 (socket says 172.21.15.6:45524) 2024-02-26T06:21:27.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 -- 172.21.15.6:0/1963581199 learned_addr learned my addr 172.21.15.6:0/1963581199 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:21:27.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e2a00700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e4075ca0 0x7ff7e419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 -- 172.21.15.6:0/1963581199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e4075ca0 msgr2=0x7ff7e419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e4075ca0 0x7ff7e419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 -- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40746d0 msgr2=0x7ff7e4199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40746d0 0x7ff7e4199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 -- 172.21.15.6:0/1963581199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40095f0 con 0x7ff7e4109a90 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.957+0000 7ff7e3e00700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e41a1440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009f60 tx=0x7ff7dc004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.958+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00e040 con 0x7ff7e4109a90 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.958+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7dc004e70 con 0x7ff7e4109a90 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.958+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc005280 con 0x7ff7e4109a90 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.958+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc009c40 con 0x7ff7e4109a90 2024-02-26T06:21:27.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.958+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e41a2190 con 0x7ff7e4109a90 2024-02-26T06:21:27.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.959+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4108850 con 0x7ff7e4109a90 2024-02-26T06:21:27.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.961+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff7dc003a00 con 0x7ff7e4109a90 2024-02-26T06:21:27.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.962+0000 7ff7dbe00700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7c80a21d0 0x7ff7c80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:21:27.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.962+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7ff7dc0ce230 con 0x7ff7e4109a90 2024-02-26T06:21:27.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.962+0000 7ff7e3400700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7c80a21d0 0x7ff7c80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:21:27.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.963+0000 7ff7e3400700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7c80a21d0 0x7ff7c80a4690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4076080 tx=0x7ff7d4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:21:27.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:27.964+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7dc0980b0 con 0x7ff7e4109a90 2024-02-26T06:21:28.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:28.200+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7e402d8a0 con 0x7ff7c80a21d0 2024-02-26T06:21:28.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:28 smithi167 ceph-mon[36229]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T06:21:28.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:21:28.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:21:28.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:28.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:28 smithi090 ceph-mon[35681]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T06:21:28.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:21:28.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:21:28.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:28.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:28 smithi006 ceph-mon[31790]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T06:21:28.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:21:28.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:21:28.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:29.727 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:29 smithi006 ceph-mon[31790]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:21:29.728 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:29 smithi006 ceph-mon[31790]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:21:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:29 smithi167 ceph-mon[36229]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:21:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:29 smithi167 ceph-mon[36229]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:21:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:29 smithi090 ceph-mon[35681]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T06:21:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:29 smithi090 ceph-mon[35681]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:21:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:31 smithi167 ceph-mon[36229]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-26T06:21:31.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:31 smithi090 ceph-mon[35681]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-26T06:21:31.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:31 smithi006 ceph-mon[31790]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-26T06:21:32.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:32 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4201336047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f9e1699-08b0-422a-9875-5a840fa6c47a"}]: dispatch 2024-02-26T06:21:32.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:32 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4201336047' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f9e1699-08b0-422a-9875-5a840fa6c47a"}]': finished 2024-02-26T06:21:32.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:32 smithi167 ceph-mon[36229]: osdmap e43: 7 total, 6 up, 7 in 2024-02-26T06:21:32.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:32 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:32.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:32 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4201336047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f9e1699-08b0-422a-9875-5a840fa6c47a"}]: dispatch 2024-02-26T06:21:32.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:32 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4201336047' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f9e1699-08b0-422a-9875-5a840fa6c47a"}]': finished 2024-02-26T06:21:32.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:32 smithi090 ceph-mon[35681]: osdmap e43: 7 total, 6 up, 7 in 2024-02-26T06:21:32.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:32 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:32.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:32 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4201336047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f9e1699-08b0-422a-9875-5a840fa6c47a"}]: dispatch 2024-02-26T06:21:32.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:32 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4201336047' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f9e1699-08b0-422a-9875-5a840fa6c47a"}]': finished 2024-02-26T06:21:32.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:32 smithi006 ceph-mon[31790]: osdmap e43: 7 total, 6 up, 7 in 2024-02-26T06:21:32.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:32 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:33 smithi090 ceph-mon[35681]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-26T06:21:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:33 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1171373099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:21:33.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:33 smithi006 ceph-mon[31790]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-26T06:21:33.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:33 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1171373099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:21:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:33 smithi167 ceph-mon[36229]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-26T06:21:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:33 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1171373099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:21:35.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:35 smithi090 ceph-mon[35681]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:21:35.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:35 smithi006 ceph-mon[31790]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:21:35.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:35 smithi167 ceph-mon[36229]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-26T06:21:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:37 smithi167 ceph-mon[36229]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:37.852 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:37 smithi090 ceph-mon[35681]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:37.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:37 smithi006 ceph-mon[31790]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:39.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:39 smithi006 ceph-mon[31790]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:39.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:39 smithi090 ceph-mon[35681]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:39.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:39 smithi167 ceph-mon[36229]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:41.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:41 smithi006 ceph-mon[31790]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:41.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:41 smithi090 ceph-mon[35681]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:41.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:41 smithi167 ceph-mon[36229]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:42.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T06:21:42.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:42.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T06:21:42.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:42.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T06:21:42.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:43.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:43 smithi006 ceph-mon[31790]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:43.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:43 smithi006 ceph-mon[31790]: Deploying daemon osd.6 on smithi006 2024-02-26T06:21:43.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:43 smithi090 ceph-mon[35681]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:43.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:43 smithi090 ceph-mon[35681]: Deploying daemon osd.6 on smithi006 2024-02-26T06:21:43.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:43 smithi167 ceph-mon[36229]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:43.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:43 smithi167 ceph-mon[36229]: Deploying daemon osd.6 on smithi006 2024-02-26T06:21:45.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:45 smithi006 ceph-mon[31790]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:45.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:45 smithi090 ceph-mon[35681]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:45.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:45 smithi167 ceph-mon[36229]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:47.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:47 smithi090 ceph-mon[35681]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:47 smithi006 ceph-mon[31790]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:47.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:47 smithi167 ceph-mon[36229]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:49.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:49 smithi006 ceph-mon[31790]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:49.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:49 smithi090 ceph-mon[35681]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:49.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:49 smithi167 ceph-mon[36229]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:51 smithi090 ceph-mon[35681]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:51 smithi006 ceph-mon[31790]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:51.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:51 smithi167 ceph-mon[36229]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:52.897 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:52.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:52.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:52.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:52.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:52.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:53.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:53 smithi006 ceph-mon[31790]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:53.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:53 smithi167 ceph-mon[36229]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:54.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:53 smithi090 ceph-mon[35681]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:55.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:55 smithi006 ceph-mon[31790]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:55.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:55 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:55.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:55 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:55.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:55 smithi006 ceph-mon[31790]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:21:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:55 smithi167 ceph-mon[36229]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:55.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:55 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:55.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:55 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:55.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:55 smithi167 ceph-mon[36229]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:21:56.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:55 smithi090 ceph-mon[35681]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:56.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:55 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:56.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:55 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:56.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:55 smithi090 ceph-mon[35681]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T06:21:56.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.914+0000 7ff7dbe00700 1 -- 172.21.15.6:0/1963581199 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff7e402d8a0 con 0x7ff7c80a21d0 2024-02-26T06:21:56.917 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 6 on host 'smithi006' 2024-02-26T06:21:56.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.919+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7c80a21d0 msgr2=0x7ff7c80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:56.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.919+0000 7ff7e9200700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7c80a21d0 0x7ff7c80a4690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4076080 tx=0x7ff7d4009310 comp rx=0 tx=0).stop 2024-02-26T06:21:56.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.919+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 msgr2=0x7ff7e41a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:21:56.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.919+0000 7ff7e9200700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e41a1440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009f60 tx=0x7ff7dc004c80 comp rx=0 tx=0).stop 2024-02-26T06:21:56.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 shutdown_connections 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7c80a21d0 0x7ff7c80a4690 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7e4109a90 0x7ff7e41a1440 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40746d0 0x7ff7e4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 --2- 172.21.15.6:0/1963581199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e4075ca0 0x7ff7e419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 >> 172.21.15.6:0/1963581199 conn(0x7ff7e40fd180 msgr2=0x7ff7e4105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 shutdown_connections 2024-02-26T06:21:56.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:21:56.920+0000 7ff7e9200700 1 -- 172.21.15.6:0/1963581199 wait complete. 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:57.741 DEBUG:teuthology.orchestra.run.smithi006:osd.6> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.6.service 2024-02-26T06:21:57.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:57.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T06:21:57.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:57.746 INFO:tasks.cephadm:Deploying osd.7 on smithi090 with /dev/vg_nvme/lv_2... 2024-02-26T06:21:57.747 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T06:21:57.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:21:57.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:21:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:21:58.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:58 smithi006 ceph-mon[31790]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:21:58.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:58 smithi006 ceph-mon[31790]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T06:21:58.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:58.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:58.453 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:21:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:21:58.424+0000 7f44a5c00700 -1 osd.6 0 waiting for initial osdmap 2024-02-26T06:21:58.661 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:58 smithi090 ceph-mon[35681]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:21:58.661 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:58 smithi090 ceph-mon[35681]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T06:21:58.661 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:58.661 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:58.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:58 smithi167 ceph-mon[36229]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:21:58.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:58 smithi167 ceph-mon[36229]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T06:21:58.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:58.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:58.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:21:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:21:58.462+0000 7f449c400700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:21:59.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:59 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:21:59.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:59 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:21:59.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:59 smithi090 ceph-mon[35681]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:59.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:59 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:59.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:21:59 smithi090 ceph-mon[35681]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' 2024-02-26T06:21:59.651 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:59 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:21:59.651 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:59 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:21:59.652 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:59 smithi006 ceph-mon[31790]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:59.652 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:59 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:59.652 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:21:59 smithi006 ceph-mon[31790]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' 2024-02-26T06:21:59.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:59 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:21:59.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:59 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:21:59.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:59 smithi167 ceph-mon[36229]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:21:59.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:59 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:21:59.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:21:59 smithi167 ceph-mon[36229]: from='osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798]' entity='osd.6' 2024-02-26T06:21:59.766 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T06:21:59.766 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T06:21:59.766 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-02-26T06:21:59.766 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-02-26T06:21:59.766 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115811 s, 905 MB/s 2024-02-26T06:21:59.766 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-02-26T06:22:00.365 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-02-26T06:22:00.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:22:00.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:00 smithi090 ceph-mon[35681]: osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] boot 2024-02-26T06:22:00.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:00 smithi090 ceph-mon[35681]: osdmap e46: 7 total, 7 up, 7 in 2024-02-26T06:22:00.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:22:00.653 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:22:00.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:22:00.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:00 smithi006 ceph-mon[31790]: osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] boot 2024-02-26T06:22:00.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:00 smithi006 ceph-mon[31790]: osdmap e46: 7 total, 7 up, 7 in 2024-02-26T06:22:00.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:22:00.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:22:00.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:00 smithi167 ceph-mon[36229]: osd.6 [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] boot 2024-02-26T06:22:00.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:00 smithi167 ceph-mon[36229]: osdmap e46: 7 total, 7 up, 7 in 2024-02-26T06:22:00.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:22:01.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:01 smithi006 ceph-mon[31790]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:22:01.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:01 smithi006 ceph-mon[31790]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T06:22:01.730 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:01 smithi090 ceph-mon[35681]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:22:01.730 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:01 smithi090 ceph-mon[35681]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T06:22:01.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:01 smithi167 ceph-mon[36229]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-26T06:22:01.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:01 smithi167 ceph-mon[36229]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T06:22:02.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 -- 172.21.15.90:0/978065725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 msgr2=0x7f14601020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 --2- 172.21.15.90:0/978065725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f14601020b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14500099c0 tx=0x7f1450009cd0 comp rx=0 tx=0).stop 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 -- 172.21.15.90:0/978065725 shutdown_connections 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 --2- 172.21.15.90:0/978065725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1460103830 0x7f146010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 --2- 172.21.15.90:0/978065725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f14601020b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 --2- 172.21.15.90:0/978065725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460102e90 0x7f14601032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 -- 172.21.15.90:0/978065725 >> 172.21.15.90:0/978065725 conn(0x7f14600fd250 msgr2=0x7f14600ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.221+0000 7f14687de700 1 -- 172.21.15.90:0/978065725 shutdown_connections 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 -- 172.21.15.90:0/978065725 wait complete. 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 Processor -- start 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 -- start start 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f1460199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460102e90 0x7f146019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:02.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1460103830 0x7f14601a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14601a1a30 con 0x7f1460103830 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14601a1b70 con 0x7f1460101c90 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f14687de700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14601a1d00 con 0x7f1460102e90 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f146657a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f1460199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f146657a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f1460199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36854/0 (socket says 172.21.15.90:36854) 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.222+0000 7f146657a700 1 -- 172.21.15.90:0/3148416898 learned_addr learned my addr 172.21.15.90:0/3148416898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f1465d79700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460102e90 0x7f146019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f146657a700 1 -- 172.21.15.90:0/3148416898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460102e90 msgr2=0x7f146019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f146657a700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460102e90 0x7f146019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f146657a700 1 -- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1460103830 msgr2=0x7f14601a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f146657a700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1460103830 0x7f14601a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f146657a700 1 -- 172.21.15.90:0/3148416898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500095f0 con 0x7f1460101c90 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f146657a700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f1460199ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1450003890 tx=0x7f145000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145001d070 con 0x7f1460101c90 2024-02-26T06:22:02.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14601a1f80 con 0x7f1460101c90 2024-02-26T06:22:02.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f145000fe50 con 0x7f1460101c90 2024-02-26T06:22:02.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450005600 con 0x7f1460101c90 2024-02-26T06:22:02.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.223+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14601a2410 con 0x7f1460101c90 2024-02-26T06:22:02.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.224+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146010d650 con 0x7f1460101c90 2024-02-26T06:22:02.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.225+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1450003930 con 0x7f1460101c90 2024-02-26T06:22:02.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.226+0000 7f14577fe700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f144c0a2150 0x7f144c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:02.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.226+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f14500cdf10 con 0x7f1460101c90 2024-02-26T06:22:02.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.226+0000 7f1465d79700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f144c0a2150 0x7f144c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:02.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.227+0000 7f1465d79700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f144c0a2150 0x7f144c0a4610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f145c009c60 tx=0x7f145c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:02.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.230+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1450097c80 con 0x7f1460101c90 2024-02-26T06:22:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:02.432+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f14600755c0 con 0x7f144c0a2150 2024-02-26T06:22:02.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:02.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:02.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:02.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:02.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:02.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:03.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:03.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:03.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi006 2024-02-26T06:22:03.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi006 to 5698M 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:03.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:03 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi006 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi006 to 5698M 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:03.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:03 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi006 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:03.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi006 to 5698M 2024-02-26T06:22:03.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:03.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:03.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:03.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:03 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:05.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:05 smithi167 ceph-mon[36229]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:05.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:05 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/1222235551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]: dispatch 2024-02-26T06:22:05.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:05 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]: dispatch 2024-02-26T06:22:05.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:05 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]': finished 2024-02-26T06:22:05.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:05 smithi167 ceph-mon[36229]: osdmap e48: 8 total, 7 up, 8 in 2024-02-26T06:22:05.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:05 smithi090 ceph-mon[35681]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:05 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/1222235551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]: dispatch 2024-02-26T06:22:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:05 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]: dispatch 2024-02-26T06:22:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:05 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]': finished 2024-02-26T06:22:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:05 smithi090 ceph-mon[35681]: osdmap e48: 8 total, 7 up, 8 in 2024-02-26T06:22:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:05 smithi006 ceph-mon[31790]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:05 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/1222235551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]: dispatch 2024-02-26T06:22:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:05 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]: dispatch 2024-02-26T06:22:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:05 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635"}]': finished 2024-02-26T06:22:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:05 smithi006 ceph-mon[31790]: osdmap e48: 8 total, 7 up, 8 in 2024-02-26T06:22:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:06.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:06 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/2031997008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:22:06.753 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:06 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/2031997008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:22:06.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:06 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/2031997008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:22:07.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:07 smithi167 ceph-mon[36229]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:07.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:07 smithi090 ceph-mon[35681]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:07.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:07 smithi006 ceph-mon[31790]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:09.573 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:09 smithi090 ceph-mon[35681]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:09.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:09 smithi167 ceph-mon[36229]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:09.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:09 smithi006 ceph-mon[31790]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:11 smithi167 ceph-mon[36229]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:11.821 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:11 smithi090 ceph-mon[35681]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:11.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:11 smithi006 ceph-mon[31790]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:12.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T06:22:12.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:12.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T06:22:12.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:12.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T06:22:12.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:13.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:13 smithi090 ceph-mon[35681]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:13.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:13 smithi090 ceph-mon[35681]: Deploying daemon osd.7 on smithi090 2024-02-26T06:22:13.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:13 smithi006 ceph-mon[31790]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:13.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:13 smithi006 ceph-mon[31790]: Deploying daemon osd.7 on smithi090 2024-02-26T06:22:13.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:13 smithi167 ceph-mon[36229]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:13.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:13 smithi167 ceph-mon[36229]: Deploying daemon osd.7 on smithi090 2024-02-26T06:22:15.796 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:15 smithi090 ceph-mon[35681]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:15.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:15 smithi006 ceph-mon[31790]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:15.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:15 smithi167 ceph-mon[36229]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:17.551 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:17 smithi090 ceph-mon[35681]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:17.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:17 smithi006 ceph-mon[31790]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:17.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:17 smithi167 ceph-mon[36229]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:19.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:19.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:19.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:19.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:19.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:19.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:19.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:19.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:19.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:19.922 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:19 smithi090 ceph-mon[35681]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:20.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:19 smithi006 ceph-mon[31790]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:19 smithi167 ceph-mon[36229]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:21.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.466+0000 7f14577fe700 1 -- 172.21.15.90:0/3148416898 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f14600755c0 con 0x7f144c0a2150 2024-02-26T06:22:21.467 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 7 on host 'smithi090' 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.468+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f144c0a2150 msgr2=0x7f144c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.468+0000 7f14687de700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f144c0a2150 0x7f144c0a4610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f145c009c60 tx=0x7f145c009260 comp rx=0 tx=0).stop 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.468+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 msgr2=0x7f1460199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.468+0000 7f14687de700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f1460199ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1450003890 tx=0x7f145000f690 comp rx=0 tx=0).stop 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 shutdown_connections 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f144c0a2150 0x7f144c0a4610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1460103830 0x7f14601a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1460101c90 0x7f1460199ea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 --2- 172.21.15.90:0/3148416898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460102e90 0x7f146019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 >> 172.21.15.90:0/3148416898 conn(0x7f14600fd250 msgr2=0x7f1460073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 shutdown_connections 2024-02-26T06:22:21.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:22:21.469+0000 7f14687de700 1 -- 172.21.15.90:0/3148416898 wait complete. 2024-02-26T06:22:21.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:22:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:21 smithi167 ceph-mon[36229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.045 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.046 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:22:22.046 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:21 smithi090 ceph-mon[35681]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:22:22.069 DEBUG:teuthology.orchestra.run.smithi090:osd.7> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.7.service 2024-02-26T06:22:22.072 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-02-26T06:22:22.072 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:22.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:22:22.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:21 smithi006 ceph-mon[31790]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T06:22:22.322 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:22:23.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:23 smithi167 ceph-mon[36229]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:23.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:23 smithi167 ceph-mon[36229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T06:22:23.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:23 smithi167 ceph-mon[36229]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T06:22:23.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:23 smithi167 ceph-mon[36229]: from='osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:22:23.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:23.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:23 smithi167 ceph-mon[36229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:22:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:23 smithi090 ceph-mon[35681]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:23 smithi090 ceph-mon[35681]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T06:22:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:23 smithi090 ceph-mon[35681]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T06:22:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:23 smithi090 ceph-mon[35681]: from='osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:22:23.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:23 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:23.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:23 smithi090 ceph-mon[35681]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:22:23.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:23 smithi006 ceph-mon[31790]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:23.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:23 smithi006 ceph-mon[31790]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T06:22:23.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:23 smithi006 ceph-mon[31790]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T06:22:23.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:23 smithi006 ceph-mon[31790]: from='osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:22:23.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:23.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:23 smithi006 ceph-mon[31790]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134535 s, 779 MB/s 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 2024-02-26T06:22:24.179 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-02-26T06:22:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-mon[35681]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:22:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-mon[35681]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T06:22:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-mon[35681]: from='osd.7 ' entity='osd.7' 2024-02-26T06:22:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:24.616 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:22:24.341+0000 7f434c7e8700 -1 osd.7 0 waiting for initial osdmap 2024-02-26T06:22:24.616 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:22:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:22:24.355+0000 7f4344dd4700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:22:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:24 smithi167 ceph-mon[36229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:22:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:24 smithi167 ceph-mon[36229]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T06:22:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:24 smithi167 ceph-mon[36229]: from='osd.7 ' entity='osd.7' 2024-02-26T06:22:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:24 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:24.886 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-02-26T06:22:24.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:24 smithi006 ceph-mon[31790]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:22:24.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:24 smithi006 ceph-mon[31790]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T06:22:24.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:24 smithi006 ceph-mon[31790]: from='osd.7 ' entity='osd.7' 2024-02-26T06:22:24.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:24 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:25.129 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:22:25.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:22:25.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:22:25.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:25.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] boot 2024-02-26T06:22:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: osdmap e51: 8 total, 8 up, 8 in 2024-02-26T06:22:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T06:22:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:25.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:25.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:22:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:22:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] boot 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: osdmap e51: 8 total, 8 up, 8 in 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:25.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:22:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:22:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-02-26T06:22:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: osd.7 [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] boot 2024-02-26T06:22:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: osdmap e51: 8 total, 8 up, 8 in 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:25.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:26.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.789+0000 7fa56d79c700 1 -- 172.21.15.167:0/2189487910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568101db0 msgr2=0x7fa568102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:26.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.789+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2189487910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568101db0 0x7fa568102210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009990 tx=0x7fa55c009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:26.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.789+0000 7fa56d79c700 1 -- 172.21.15.167:0/2189487910 shutdown_connections 2024-02-26T06:22:26.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.789+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2189487910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa568102750 0x7fa568109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:26.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2189487910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa568100bb0 0x7fa568100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:26.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2189487910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568101db0 0x7fa568102210 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 -- 172.21.15.167:0/2189487910 >> 172.21.15.167:0/2189487910 conn(0x7fa5680fc170 msgr2=0x7fa5680fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 -- 172.21.15.167:0/2189487910 shutdown_connections 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 -- 172.21.15.167:0/2189487910 wait complete. 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 Processor -- start 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.790+0000 7fa56d79c700 1 -- start start 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 0x7fa568197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa568101db0 0x7fa568198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa568102750 0x7fa56819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56819f950 con 0x7fa568101db0 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56819fa90 con 0x7fa568102750 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56819fc20 con 0x7fa568100bb0 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 0x7fa568197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 0x7fa568197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44444/0 (socket says 172.21.15.167:44444) 2024-02-26T06:22:26.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 -- 172.21.15.167:0/2504973694 learned_addr learned my addr 172.21.15.167:0/2504973694 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa5677fe700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa568102750 0x7fa56819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 -- 172.21.15.167:0/2504973694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa568102750 msgr2=0x7fa56819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa568102750 0x7fa56819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 -- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa568101db0 msgr2=0x7fa568198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa568101db0 0x7fa568198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 -- 172.21.15.167:0/2504973694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0095f0 con 0x7fa568100bb0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa566ffd700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 0x7fa568197dc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa55800b530 tx=0x7fa55800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558011830 con 0x7fa568100bb0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56819ff00 con 0x7fa568100bb0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa558011e70 con 0x7fa568100bb0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.791+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800f2e0 con 0x7fa568100bb0 2024-02-26T06:22:26.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.792+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5681a03a0 con 0x7fa568100bb0 2024-02-26T06:22:26.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.793+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa568068930 con 0x7fa568100bb0 2024-02-26T06:22:26.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.794+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fa5580105f0 con 0x7fa568100bb0 2024-02-26T06:22:26.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.794+0000 7fa54ffff700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa5500a2150 0x7fa5500a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:26.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.794+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5397+0+0 (secure 0 0 0) 0x7fa5580ccba0 con 0x7fa568100bb0 2024-02-26T06:22:26.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.795+0000 7fa5667fc700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa5500a2150 0x7fa5500a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:26.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.795+0000 7fa5667fc700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa5500a2150 0x7fa5500a4610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0095c0 tx=0x7fa55c009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:26.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:26.799+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5580966d0 con 0x7fa568100bb0 2024-02-26T06:22:26.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:26 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi090 2024-02-26T06:22:26.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:26 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi090 to 5693M 2024-02-26T06:22:26.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:26 smithi167 ceph-mon[36229]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T06:22:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:27.000+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa56802d910 con 0x7fa5500a2150 2024-02-26T06:22:27.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:26 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi090 2024-02-26T06:22:27.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:26 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi090 to 5693M 2024-02-26T06:22:27.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:26 smithi090 ceph-mon[35681]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T06:22:27.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:26 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi090 2024-02-26T06:22:27.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:26 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi090 to 5693M 2024-02-26T06:22:27.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:26 smithi006 ceph-mon[31790]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T06:22:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:27 smithi167 ceph-mon[36229]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:27 smithi167 ceph-mon[36229]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-02-26T06:22:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:27 smithi167 ceph-mon[36229]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T06:22:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:27.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:27 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:27 smithi090 ceph-mon[35681]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:27 smithi090 ceph-mon[35681]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-02-26T06:22:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:27 smithi090 ceph-mon[35681]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T06:22:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:27 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:27 smithi006 ceph-mon[31790]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:27 smithi006 ceph-mon[31790]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-02-26T06:22:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:27 smithi006 ceph-mon[31790]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T06:22:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:27 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:28.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:28 smithi167 ceph-mon[36229]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:28.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:28 smithi167 ceph-mon[36229]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T06:22:29.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:28 smithi090 ceph-mon[35681]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:29.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:28 smithi090 ceph-mon[35681]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T06:22:29.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:28 smithi006 ceph-mon[31790]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:29.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:28 smithi006 ceph-mon[31790]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T06:22:30.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:29 smithi167 ceph-mon[36229]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:30.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:29 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/2100066920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]: dispatch 2024-02-26T06:22:30.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:29 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]: dispatch 2024-02-26T06:22:30.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:29 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]': finished 2024-02-26T06:22:30.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:29 smithi167 ceph-mon[36229]: osdmap e56: 9 total, 8 up, 9 in 2024-02-26T06:22:30.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:29 smithi090 ceph-mon[35681]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:29 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/2100066920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]: dispatch 2024-02-26T06:22:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:29 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]: dispatch 2024-02-26T06:22:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:29 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]': finished 2024-02-26T06:22:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:29 smithi090 ceph-mon[35681]: osdmap e56: 9 total, 8 up, 9 in 2024-02-26T06:22:30.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:30.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:29 smithi006 ceph-mon[31790]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:29 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/2100066920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]: dispatch 2024-02-26T06:22:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:29 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]: dispatch 2024-02-26T06:22:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:29 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863e848b-67fd-4bc7-82c5-b3b69df061b9"}]': finished 2024-02-26T06:22:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:29 smithi006 ceph-mon[31790]: osdmap e56: 9 total, 8 up, 9 in 2024-02-26T06:22:30.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:31.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:30 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/2953882494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:22:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:30 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/2953882494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:22:31.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:30 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/2953882494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:22:32.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:31 smithi090 ceph-mon[35681]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:31 smithi006 ceph-mon[31790]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:32.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:31 smithi167 ceph-mon[36229]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:33.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:32 smithi090 ceph-mon[35681]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-02-26T06:22:33.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:32 smithi090 ceph-mon[35681]: Cluster is now healthy 2024-02-26T06:22:33.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:32 smithi006 ceph-mon[31790]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-02-26T06:22:33.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:32 smithi006 ceph-mon[31790]: Cluster is now healthy 2024-02-26T06:22:33.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:32 smithi167 ceph-mon[36229]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-02-26T06:22:33.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:32 smithi167 ceph-mon[36229]: Cluster is now healthy 2024-02-26T06:22:33.993 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:33 smithi167 ceph-mon[36229]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:34.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:33 smithi006 ceph-mon[31790]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:33 smithi090 ceph-mon[35681]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:36.167 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:35 smithi090 ceph-mon[35681]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:36.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:35 smithi006 ceph-mon[31790]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:36.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:35 smithi167 ceph-mon[36229]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:38.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:37 smithi167 ceph-mon[36229]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:38.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T06:22:38.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:38.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:37 smithi006 ceph-mon[31790]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:38.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T06:22:38.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:38.361 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:37 smithi090 ceph-mon[35681]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:38.361 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T06:22:38.361 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:39.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:38 smithi006 ceph-mon[31790]: Deploying daemon osd.8 on smithi167 2024-02-26T06:22:39.188 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:38 smithi167 ceph-mon[36229]: Deploying daemon osd.8 on smithi167 2024-02-26T06:22:39.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:38 smithi090 ceph-mon[35681]: Deploying daemon osd.8 on smithi167 2024-02-26T06:22:40.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:39 smithi167 ceph-mon[36229]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:40.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:39 smithi006 ceph-mon[31790]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:39 smithi090 ceph-mon[35681]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:42.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:41 smithi006 ceph-mon[31790]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:41 smithi167 ceph-mon[36229]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:41 smithi090 ceph-mon[35681]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:44.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:43 smithi090 ceph-mon[35681]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:44.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:44.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:44.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:44.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:43 smithi006 ceph-mon[31790]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:44.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:43 smithi167 ceph-mon[36229]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:46.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:45 smithi090 ceph-mon[35681]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:46.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:45 smithi006 ceph-mon[31790]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:46.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:45 smithi167 ceph-mon[36229]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:46.734 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-02-26T06:22:46.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.734+0000 7fa54ffff700 1 -- 172.21.15.167:0/2504973694 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa56802d910 con 0x7fa5500a2150 2024-02-26T06:22:46.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa5500a2150 msgr2=0x7fa5500a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa5500a2150 0x7fa5500a4610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0095c0 tx=0x7fa55c009510 comp rx=0 tx=0).stop 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 msgr2=0x7fa568197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 0x7fa568197dc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa55800b530 tx=0x7fa55800b840 comp rx=0 tx=0).stop 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 shutdown_connections 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa5500a2150 0x7fa5500a4610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa568102750 0x7fa56819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa568100bb0 0x7fa568197dc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 --2- 172.21.15.167:0/2504973694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa568101db0 0x7fa568198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.737+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 >> 172.21.15.167:0/2504973694 conn(0x7fa5680fc170 msgr2=0x7fa568105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.738+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 shutdown_connections 2024-02-26T06:22:46.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:22:46.738+0000 7fa56d79c700 1 -- 172.21.15.167:0/2504973694 wait complete. 2024-02-26T06:22:46.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:46.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:22:46.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:22:47.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:22:47.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T06:22:47.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:22:47.380 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.8.service 2024-02-26T06:22:47.382 INFO:tasks.cephadm:Deploying osd.9 on smithi006 with /dev/vg_nvme/lv_1... 2024-02-26T06:22:47.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T06:22:47.854 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:47.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:47 smithi167 ceph-mon[36229]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:47.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:47 smithi167 ceph-mon[36229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T06:22:47.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:47 smithi167 ceph-mon[36229]: from='osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:22:47.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:47 smithi167 ceph-mon[36229]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T06:22:47.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:47.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:47 smithi167 ceph-mon[36229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:22:48.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:47 smithi006 ceph-mon[31790]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:48.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:47 smithi006 ceph-mon[31790]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T06:22:48.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:47 smithi006 ceph-mon[31790]: from='osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:22:48.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:47 smithi006 ceph-mon[31790]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T06:22:48.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:48.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:47 smithi006 ceph-mon[31790]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:22:48.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:47 smithi090 ceph-mon[35681]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:48.492 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:47 smithi090 ceph-mon[35681]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T06:22:48.492 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:47 smithi090 ceph-mon[35681]: from='osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:22:48.492 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:47 smithi090 ceph-mon[35681]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T06:22:48.492 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:48.492 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:47 smithi090 ceph-mon[35681]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:22:49.904 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-mon[36229]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:49.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-mon[36229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:22:49.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-mon[36229]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T06:22:49.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:49.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:49.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-mon[36229]: from='osd.8 ' entity='osd.8' 2024-02-26T06:22:49.905 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:22:49.571+0000 7f4289953700 -1 osd.8 0 waiting for initial osdmap 2024-02-26T06:22:49.905 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:22:49 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:22:49.593+0000 7f4281f3f700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:22:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:49 smithi090 ceph-mon[35681]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:49 smithi090 ceph-mon[35681]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:22:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:49 smithi090 ceph-mon[35681]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T06:22:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:49 smithi090 ceph-mon[35681]: from='osd.8 ' entity='osd.8' 2024-02-26T06:22:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:49 smithi006 ceph-mon[31790]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-26T06:22:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:49 smithi006 ceph-mon[31790]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:22:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:49 smithi006 ceph-mon[31790]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T06:22:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:49 smithi006 ceph-mon[31790]: from='osd.8 ' entity='osd.8' 2024-02-26T06:22:50.663 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T06:22:50.663 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T06:22:50.663 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-02-26T06:22:50.663 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-02-26T06:22:50.663 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.035251 s, 297 MB/s 2024-02-26T06:22:50.663 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] boot 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: osdmap e59: 9 total, 9 up, 9 in 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:50.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] boot 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: osdmap e59: 9 total, 9 up, 9 in 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:50.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:50.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:50.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:51.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:22:51.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:22:51.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: osd.8 [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] boot 2024-02-26T06:22:51.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: osdmap e59: 9 total, 9 up, 9 in 2024-02-26T06:22:51.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:22:51.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:22:51.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:22:51.520 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_1 2024-02-26T06:22:51.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:51 smithi167 ceph-mon[36229]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:51.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:51 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi167 2024-02-26T06:22:51.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:51 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi167 to 5693M 2024-02-26T06:22:51.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:51 smithi167 ceph-mon[36229]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T06:22:51.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:51 smithi167 ceph-mon[36229]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T06:22:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:22:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:51 smithi090 ceph-mon[35681]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:51 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi167 2024-02-26T06:22:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:51 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi167 to 5693M 2024-02-26T06:22:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:51 smithi090 ceph-mon[35681]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T06:22:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:51 smithi090 ceph-mon[35681]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T06:22:52.164 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:51 smithi006 ceph-mon[31790]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:52.164 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:51 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi167 2024-02-26T06:22:52.164 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:51 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi167 to 5693M 2024-02-26T06:22:52.164 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:51 smithi006 ceph-mon[31790]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T06:22:52.164 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:51 smithi006 ceph-mon[31790]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T06:22:54.020 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:53 smithi006 ceph-mon[31790]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:54.020 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:53 smithi006 ceph-mon[31790]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T06:22:54.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:53 smithi090 ceph-mon[35681]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:54.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:53 smithi090 ceph-mon[35681]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.196+0000 7fbc5de00700 1 -- 172.21.15.6:0/3094296132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58102ed0 msgr2=0x7fbc58103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.196+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3094296132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58102ed0 0x7fbc58103330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009990 tx=0x7fbc44009ca0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 -- 172.21.15.6:0/3094296132 shutdown_connections 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3094296132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc58103870 0x7fbc5810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3094296132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58101cd0 0x7fbc581020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3094296132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58102ed0 0x7fbc58103330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 -- 172.21.15.6:0/3094296132 >> 172.21.15.6:0/3094296132 conn(0x7fbc580fd270 msgr2=0x7fbc580ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 -- 172.21.15.6:0/3094296132 shutdown_connections 2024-02-26T06:22:54.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.197+0000 7fbc5de00700 1 -- 172.21.15.6:0/3094296132 wait complete. 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.198+0000 7fbc5de00700 1 Processor -- start 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.198+0000 7fbc5de00700 1 -- start start 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.198+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58101cd0 0x7fbc58197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 0x7fbc581981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc58103870 0x7fbc5819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc5de00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819f7f0 con 0x7fbc58102ed0 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc5de00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819f930 con 0x7fbc58103870 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc5de00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819faf0 con 0x7fbc58101cd0 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 0x7fbc581981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58101cd0 0x7fbc58197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 0x7fbc581981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51514/0 (socket says 172.21.15.6:51514) 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56000700 1 -- 172.21.15.6:0/3167638285 learned_addr learned my addr 172.21.15.6:0/3167638285 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:22:54.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc57400700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc58103870 0x7fbc5819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56a00700 1 -- 172.21.15.6:0/3167638285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc58103870 msgr2=0x7fbc5819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56a00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc58103870 0x7fbc5819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56a00700 1 -- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 msgr2=0x7fbc581981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56a00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 0x7fbc581981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.199+0000 7fbc56a00700 1 -- 172.21.15.6:0/3167638285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009520 con 0x7fbc58101cd0 2024-02-26T06:22:54.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.200+0000 7fbc56000700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 0x7fbc581981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.200+0000 7fbc56a00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58101cd0 0x7fbc58197c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800e9c0 tx=0x7fbc4800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.200+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800cb70 con 0x7fbc58101cd0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.200+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc4800ee60 con 0x7fbc58101cd0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.200+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc440095f0 con 0x7fbc58101cd0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.200+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48018680 con 0x7fbc58101cd0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.201+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc581a0140 con 0x7fbc58101cd0 2024-02-26T06:22:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.202+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc58109090 con 0x7fbc58101cd0 2024-02-26T06:22:54.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.204+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fbc48010430 con 0x7fbc58101cd0 2024-02-26T06:22:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.205+0000 7fbc5ca00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbc3c0a2150 0x7fbc3c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:22:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.205+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fbc48014070 con 0x7fbc58101cd0 2024-02-26T06:22:54.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.205+0000 7fbc56000700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbc3c0a2150 0x7fbc3c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:22:54.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.206+0000 7fbc56000700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbc3c0a2150 0x7fbc3c0a4610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc58199180 tx=0x7fbc4400b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:22:54.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.207+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc48096ea0 con 0x7fbc58101cd0 2024-02-26T06:22:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:53 smithi167 ceph-mon[36229]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:53 smithi167 ceph-mon[36229]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T06:22:54.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:22:54.441+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc5802d870 con 0x7fbc3c0a2150 2024-02-26T06:22:55.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:54 smithi090 ceph-mon[35681]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T06:22:55.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:55.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:55.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:55.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:54 smithi167 ceph-mon[36229]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T06:22:55.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:55.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:55.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:54 smithi006 ceph-mon[31790]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T06:22:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:22:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:22:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:22:56.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:55 smithi090 ceph-mon[35681]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:56.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:55 smithi090 ceph-mon[35681]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:56.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:55 smithi006 ceph-mon[31790]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:56.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:55 smithi006 ceph-mon[31790]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:56.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:55 smithi167 ceph-mon[36229]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:22:56.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:55 smithi167 ceph-mon[36229]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:22:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:57 smithi090 ceph-mon[35681]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-02-26T06:22:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:57 smithi006 ceph-mon[31790]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-02-26T06:22:58.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:57 smithi167 ceph-mon[36229]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-02-26T06:22:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:58 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3450268947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]: dispatch 2024-02-26T06:22:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:58 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]: dispatch 2024-02-26T06:22:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:58 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]': finished 2024-02-26T06:22:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:58 smithi090 ceph-mon[35681]: osdmap e64: 10 total, 9 up, 10 in 2024-02-26T06:22:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:22:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:58 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3450268947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]: dispatch 2024-02-26T06:22:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:58 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]: dispatch 2024-02-26T06:22:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:58 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]': finished 2024-02-26T06:22:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:58 smithi006 ceph-mon[31790]: osdmap e64: 10 total, 9 up, 10 in 2024-02-26T06:22:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:22:59.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:58 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3450268947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]: dispatch 2024-02-26T06:22:59.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:58 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]: dispatch 2024-02-26T06:22:59.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:58 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ee193e3-93b3-47fb-84b4-39a14c4a060f"}]': finished 2024-02-26T06:22:59.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:58 smithi167 ceph-mon[36229]: osdmap e64: 10 total, 9 up, 10 in 2024-02-26T06:22:59.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:00.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:59 smithi090 ceph-mon[35681]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-02-26T06:23:00.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:22:59 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3031412439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:59 smithi006 ceph-mon[31790]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-02-26T06:23:00.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:22:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3031412439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:00.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:59 smithi167 ceph-mon[36229]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-02-26T06:23:00.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:22:59 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3031412439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:01 smithi090 ceph-mon[35681]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-26T06:23:02.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:01 smithi006 ceph-mon[31790]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-26T06:23:02.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:01 smithi167 ceph-mon[36229]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-26T06:23:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:03 smithi090 ceph-mon[35681]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-26T06:23:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:03 smithi006 ceph-mon[31790]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-26T06:23:04.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:03 smithi167 ceph-mon[36229]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-26T06:23:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:05 smithi090 ceph-mon[35681]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:23:06.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:05 smithi006 ceph-mon[31790]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:23:06.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:05 smithi167 ceph-mon[36229]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:23:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:07 smithi090 ceph-mon[35681]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:07 smithi006 ceph-mon[31790]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:08.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:07 smithi167 ceph-mon[36229]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:09.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T06:23:09.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:08 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:09.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T06:23:09.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:08 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:09.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T06:23:09.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:08 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:09 smithi090 ceph-mon[35681]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:09 smithi090 ceph-mon[35681]: Deploying daemon osd.9 on smithi006 2024-02-26T06:23:10.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:09 smithi006 ceph-mon[31790]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:10.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:09 smithi006 ceph-mon[31790]: Deploying daemon osd.9 on smithi006 2024-02-26T06:23:10.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:09 smithi167 ceph-mon[36229]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:10.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:09 smithi167 ceph-mon[36229]: Deploying daemon osd.9 on smithi006 2024-02-26T06:23:12.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:11 smithi090 ceph-mon[35681]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:12.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:11 smithi006 ceph-mon[31790]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:12.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:11 smithi167 ceph-mon[36229]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:13 smithi090 ceph-mon[35681]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:14.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:13 smithi006 ceph-mon[31790]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:14.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:13 smithi167 ceph-mon[36229]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:16.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:15 smithi006 ceph-mon[31790]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:16.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:15 smithi090 ceph-mon[35681]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:16.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:15 smithi167 ceph-mon[36229]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:18.169 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:17 smithi006 ceph-mon[31790]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:18.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:18.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:18.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:18.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:17 smithi167 ceph-mon[36229]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:18.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:18.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:18.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:17 smithi090 ceph-mon[35681]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:20.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:19 smithi006 ceph-mon[31790]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:19 smithi167 ceph-mon[36229]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:19 smithi090 ceph-mon[35681]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:21.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:21 smithi006 ceph-mon[31790]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:21.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:21.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:21 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:21.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:21 smithi006 ceph-mon[31790]: from='osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:23:21.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:21 smithi006 ceph-mon[31790]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:23:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:21 smithi167 ceph-mon[36229]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:21 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:21 smithi167 ceph-mon[36229]: from='osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:23:21.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:21 smithi167 ceph-mon[36229]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:23:22.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:21 smithi090 ceph-mon[35681]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:22.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:22.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:21 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:22.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:21 smithi090 ceph-mon[35681]: from='osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:23:22.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:21 smithi090 ceph-mon[35681]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T06:23:22.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.113+0000 7fbc5ca00700 1 -- 172.21.15.6:0/3167638285 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbc5802d870 con 0x7fbc3c0a2150 2024-02-26T06:23:22.116 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 9 on host 'smithi006' 2024-02-26T06:23:22.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbc3c0a2150 msgr2=0x7fbc3c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:22.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbc3c0a2150 0x7fbc3c0a4610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc58199180 tx=0x7fbc4400b7f0 comp rx=0 tx=0).stop 2024-02-26T06:23:22.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58101cd0 msgr2=0x7fbc58197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:22.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58101cd0 0x7fbc58197c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800e9c0 tx=0x7fbc4800ecd0 comp rx=0 tx=0).stop 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 shutdown_connections 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbc3c0a2150 0x7fbc3c0a4610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc58103870 0x7fbc5819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc58101cd0 0x7fbc58197c70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 --2- 172.21.15.6:0/3167638285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc58102ed0 0x7fbc581981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 >> 172.21.15.6:0/3167638285 conn(0x7fbc580fd270 msgr2=0x7fbc581061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 shutdown_connections 2024-02-26T06:23:22.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:23:22.118+0000 7fbc5de00700 1 -- 172.21.15.6:0/3167638285 wait complete. 2024-02-26T06:23:22.905 DEBUG:teuthology.orchestra.run.smithi006:osd.9> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.9.service 2024-02-26T06:23:22.911 INFO:tasks.cephadm:Deploying osd.10 on smithi090 with /dev/vg_nvme/lv_1... 2024-02-26T06:23:22.911 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: from='osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:23.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:22 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:23.149 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:23:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:23:23.005+0000 7ff114800700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:23:23.158 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: from='osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:23.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:22 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: from='osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:23.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:22 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:24.252 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:23 smithi090 ceph-mon[35681]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:24.252 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:23 smithi090 ceph-mon[35681]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:23:24.252 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:23 smithi090 ceph-mon[35681]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T06:23:24.252 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:23 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:24.252 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:23 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:24.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:23 smithi006 ceph-mon[31790]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:24.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:23 smithi006 ceph-mon[31790]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:23:24.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:23 smithi006 ceph-mon[31790]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T06:23:24.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:24.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:23 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:23 smithi167 ceph-mon[36229]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:23 smithi167 ceph-mon[36229]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-02-26T06:23:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:23 smithi167 ceph-mon[36229]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T06:23:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:24.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:23 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.098 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T06:23:25.098 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T06:23:25.098 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-02-26T06:23:25.098 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-02-26T06:23:25.098 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113196 s, 926 MB/s 2024-02-26T06:23:25.098 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] boot 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: osdmap e67: 10 total, 10 up, 10 in 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:25 smithi090 ceph-mon[35681]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:25.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:24 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:23:25.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:25 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:23:25.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:25 smithi167 ceph-mon[36229]: osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] boot 2024-02-26T06:23:25.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:25 smithi167 ceph-mon[36229]: osdmap e67: 10 total, 10 up, 10 in 2024-02-26T06:23:25.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:25 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:25 smithi167 ceph-mon[36229]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:25.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:24 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:23:25.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:25 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:23:25.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:25 smithi006 ceph-mon[31790]: osd.9 [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] boot 2024-02-26T06:23:25.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:25 smithi006 ceph-mon[31790]: osdmap e67: 10 total, 10 up, 10 in 2024-02-26T06:23:25.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:25 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:23:25.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:25 smithi006 ceph-mon[31790]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-02-26T06:23:25.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-02-26T06:23:25.934 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:23:26.291 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:26 smithi090 ceph-mon[35681]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T06:23:26.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:26 smithi006 ceph-mon[31790]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T06:23:26.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:26 smithi167 ceph-mon[36229]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T06:23:27.283 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:27 smithi090 ceph-mon[35681]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:27.353 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:27 smithi006 ceph-mon[31790]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 -- 172.21.15.90:0/1500962079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 msgr2=0x7f7960100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 --2- 172.21.15.90:0/1500962079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f7960100fc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f79500099c0 tx=0x7f7950009cd0 comp rx=0 tx=0).stop 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 -- 172.21.15.90:0/1500962079 shutdown_connections 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 --2- 172.21.15.90:0/1500962079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7960102740 0x7f7960108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 --2- 172.21.15.90:0/1500962079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f7960100fc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 --2- 172.21.15.90:0/1500962079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7960101da0 0x7f7960102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 -- 172.21.15.90:0/1500962079 >> 172.21.15.90:0/1500962079 conn(0x7f79600fc160 msgr2=0x7f79600fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 -- 172.21.15.90:0/1500962079 shutdown_connections 2024-02-26T06:23:27.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.475+0000 7f7964cd1700 1 -- 172.21.15.90:0/1500962079 wait complete. 2024-02-26T06:23:27.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 Processor -- start 2024-02-26T06:23:27.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 -- start start 2024-02-26T06:23:27.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f796006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7960101da0 0x7f796019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7960102740 0x7f79601a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79601a1a80 con 0x7f7960102740 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79601a1bf0 con 0x7f7960100ba0 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f7964cd1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79601a1db0 con 0x7f7960101da0 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f795e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f796006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f795e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f796006aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33862/0 (socket says 172.21.15.90:33862) 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.476+0000 7f795e59c700 1 -- 172.21.15.90:0/3804675281 learned_addr learned my addr 172.21.15.90:0/3804675281 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795dd9b700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7960101da0 0x7f796019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795e59c700 1 -- 172.21.15.90:0/3804675281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7960101da0 msgr2=0x7f796019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795e59c700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7960101da0 0x7f796019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795e59c700 1 -- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7960102740 msgr2=0x7f79601a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795e59c700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7960102740 0x7f79601a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795e59c700 1 -- 172.21.15.90:0/3804675281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500095f0 con 0x7f7960100ba0 2024-02-26T06:23:27.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f795e59c700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f796006aef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7950009f50 tx=0x7f7950003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:27.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795001d070 con 0x7f7960100ba0 2024-02-26T06:23:27.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79601a2030 con 0x7f7960100ba0 2024-02-26T06:23:27.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7950022470 con 0x7f7960100ba0 2024-02-26T06:23:27.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000f6c0 con 0x7f7960100ba0 2024-02-26T06:23:27.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.477+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79601a25a0 con 0x7f7960100ba0 2024-02-26T06:23:27.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.478+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960107fe0 con 0x7f7960100ba0 2024-02-26T06:23:27.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.479+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f79500225e0 con 0x7f7960100ba0 2024-02-26T06:23:27.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.480+0000 7f794f7fe700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79480a2150 0x7f79480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:27.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.480+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f79500cf0a0 con 0x7f7960100ba0 2024-02-26T06:23:27.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.480+0000 7f795dd9b700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79480a2150 0x7f79480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:27.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.481+0000 7f795dd9b700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79480a2150 0x7f79480a4610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7954006fd0 tx=0x7f7954008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:27.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.483+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7950098990 con 0x7f7960100ba0 2024-02-26T06:23:27.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:27 smithi167 ceph-mon[36229]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:27.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:27.688+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f796002d7d0 con 0x7f79480a2150 2024-02-26T06:23:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:23:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:23:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi006 2024-02-26T06:23:28.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi006 to 4273M 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:28.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:28 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi006 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi006 to 4273M 2024-02-26T06:23:28.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:28.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:28.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:28 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi006 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi006 to 4273M 2024-02-26T06:23:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:28.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:28.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:28.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:28 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:29.305 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:29 smithi090 ceph-mon[35681]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:29.305 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:29.305 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:29.306 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:29.306 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:29 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:29.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:29 smithi006 ceph-mon[31790]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:29.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:29.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:29.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:29.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:29 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:29.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:29 smithi167 ceph-mon[36229]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:29.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:29.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:29.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:29.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:29 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/2244131439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]: dispatch 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]: dispatch 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]': finished 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: osdmap e69: 11 total, 10 up, 11 in 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:31.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:31 smithi006 ceph-mon[31790]: from='client.? 172.21.15.90:0/701286911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/2244131439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]: dispatch 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]: dispatch 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]': finished 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: osdmap e69: 11 total, 10 up, 11 in 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:31 smithi167 ceph-mon[36229]: from='client.? 172.21.15.90:0/701286911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/2244131439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]: dispatch 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]: dispatch 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f22982-ab74-4fe3-b273-a2d992a543e5"}]': finished 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: osdmap e69: 11 total, 10 up, 11 in 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:31 smithi090 ceph-mon[35681]: from='client.? 172.21.15.90:0/701286911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:33 smithi006 ceph-mon[31790]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:33.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:33 smithi167 ceph-mon[36229]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:33 smithi090 ceph-mon[35681]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:35.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:35 smithi090 ceph-mon[35681]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:35.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:35 smithi006 ceph-mon[31790]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:35.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:35 smithi167 ceph-mon[36229]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:37.221 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:37 smithi090 ceph-mon[35681]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:37.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:37 smithi006 ceph-mon[31790]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:37.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:37 smithi167 ceph-mon[36229]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:38.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:38 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T06:23:38.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:38 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:38.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:38 smithi006 ceph-mon[31790]: Deploying daemon osd.10 on smithi090 2024-02-26T06:23:38.460 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:38 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T06:23:38.461 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:38 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:38.461 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:38 smithi090 ceph-mon[35681]: Deploying daemon osd.10 on smithi090 2024-02-26T06:23:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:38 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T06:23:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:38 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:38 smithi167 ceph-mon[36229]: Deploying daemon osd.10 on smithi090 2024-02-26T06:23:39.350 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:39 smithi090 ceph-mon[35681]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:39.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:39 smithi006 ceph-mon[31790]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:39.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:39 smithi167 ceph-mon[36229]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:41.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:41 smithi006 ceph-mon[31790]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:41.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:41 smithi090 ceph-mon[35681]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:41.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:41 smithi167 ceph-mon[36229]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:43 smithi167 ceph-mon[36229]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:43.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:43 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:43.537 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:43 smithi090 ceph-mon[35681]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:43.537 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:43.537 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:43.537 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:43 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:43.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:43 smithi006 ceph-mon[31790]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:43.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:43.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:43.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:43 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:45.315 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:45 smithi090 ceph-mon[35681]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:45.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:45 smithi167 ceph-mon[36229]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:45.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:45 smithi006 ceph-mon[31790]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:46.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.420+0000 7f794f7fe700 1 -- 172.21.15.90:0/3804675281 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f796002d7d0 con 0x7f79480a2150 2024-02-26T06:23:46.421 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 10 on host 'smithi090' 2024-02-26T06:23:46.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79480a2150 msgr2=0x7f79480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79480a2150 0x7f79480a4610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7954006fd0 tx=0x7f7954008040 comp rx=0 tx=0).stop 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 msgr2=0x7f796006aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f796006aef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7950009f50 tx=0x7f7950003960 comp rx=0 tx=0).stop 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 shutdown_connections 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79480a2150 0x7f79480a4610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7960102740 0x7f79601a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7960100ba0 0x7f796006aef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 --2- 172.21.15.90:0/3804675281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7960101da0 0x7f796019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 >> 172.21.15.90:0/3804675281 conn(0x7f79600fc160 msgr2=0x7f7960105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 shutdown_connections 2024-02-26T06:23:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-02-26T06:23:46.423+0000 7f7964cd1700 1 -- 172.21.15.90:0/3804675281 wait complete. 2024-02-26T06:23:46.986 DEBUG:teuthology.orchestra.run.smithi090:osd.10> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.10.service 2024-02-26T06:23:46.988 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-02-26T06:23:46.988 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T06:23:47.115 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:23:47.047+0000 7ffbd45477c0 -1 osd.10 0 log_to_monitors true 2024-02-26T06:23:47.240 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:23:47.664 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:47.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: from='osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:23:47.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:23:47.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: from='osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:23:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:23:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.814 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:47.814 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.814 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.815 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: from='osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:23:47.815 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T06:23:47.815 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:47.815 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:48.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:48 smithi006 ceph-mon[31790]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T06:23:48.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:48 smithi006 ceph-mon[31790]: from='osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:23:48.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:48 smithi006 ceph-mon[31790]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T06:23:48.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:48 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:48.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:48 smithi006 ceph-mon[31790]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:23:48.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:48 smithi167 ceph-mon[36229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T06:23:48.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:48 smithi167 ceph-mon[36229]: from='osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:23:48.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:48 smithi167 ceph-mon[36229]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T06:23:48.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:48 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:48.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:48 smithi167 ceph-mon[36229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:23:48.762 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:23:48.417+0000 7ffbc93ab700 -1 osd.10 0 waiting for initial osdmap 2024-02-26T06:23:48.762 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:23:48.433+0000 7ffbc599f700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:23:48.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-mon[35681]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T06:23:48.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-mon[35681]: from='osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:23:48.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-mon[35681]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T06:23:48.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:48.763 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:48 smithi090 ceph-mon[35681]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-02-26T06:23:49.228 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T06:23:49.229 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T06:23:49.229 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-02-26T06:23:49.229 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-02-26T06:23:49.229 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128903 s, 813 MB/s 2024-02-26T06:23:49.229 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-02-26T06:23:49.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:49 smithi167 ceph-mon[36229]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:49.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:49 smithi167 ceph-mon[36229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:23:49.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:49 smithi167 ceph-mon[36229]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T06:23:49.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:49.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:49 smithi090 ceph-mon[35681]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:49.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:49 smithi090 ceph-mon[35681]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:23:49.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:49 smithi090 ceph-mon[35681]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T06:23:49.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:49.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:49 smithi006 ceph-mon[31790]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-26T06:23:49.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:49 smithi006 ceph-mon[31790]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-02-26T06:23:49.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:49 smithi006 ceph-mon[31790]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T06:23:49.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:49.993 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-02-26T06:23:50.240 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:23:50.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:50 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:23:50.908 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:50 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:23:50.908 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:50 smithi090 ceph-mon[35681]: osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] boot 2024-02-26T06:23:50.908 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:50 smithi090 ceph-mon[35681]: osdmap e72: 11 total, 11 up, 11 in 2024-02-26T06:23:50.908 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:50.908 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:50 smithi090 ceph-mon[35681]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T06:23:50.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:50 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:23:50.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:50 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:23:50.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:50 smithi167 ceph-mon[36229]: osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] boot 2024-02-26T06:23:50.910 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:50 smithi167 ceph-mon[36229]: osdmap e72: 11 total, 11 up, 11 in 2024-02-26T06:23:50.910 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:50.910 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:50 smithi167 ceph-mon[36229]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T06:23:50.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:50 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:23:50.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:50 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:23:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:50 smithi006 ceph-mon[31790]: osd.10 [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] boot 2024-02-26T06:23:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:50 smithi006 ceph-mon[31790]: osdmap e72: 11 total, 11 up, 11 in 2024-02-26T06:23:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:23:50.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:50 smithi006 ceph-mon[31790]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi090 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi090 to 4270M 2024-02-26T06:23:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:51.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi090 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi090 to 4270M 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:51.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 -- 172.21.15.167:0/4217380421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 msgr2=0x7f7acc107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:51.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/4217380421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc107440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009990 tx=0x7f7ab4009ca0 comp rx=0 tx=0).stop 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 -- 172.21.15.167:0/4217380421 shutdown_connections 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/4217380421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 0x7f7acc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/4217380421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7acc102720 0x7f7acc104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/4217380421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc107440 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 -- 172.21.15.167:0/4217380421 >> 172.21.15.167:0/4217380421 conn(0x7f7acc0fc130 msgr2=0x7f7acc0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 -- 172.21.15.167:0/4217380421 shutdown_connections 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.841+0000 7f7ad18f1700 1 -- 172.21.15.167:0/4217380421 wait complete. 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 Processor -- start 2024-02-26T06:23:51.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 -- start start 2024-02-26T06:23:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7acc102720 0x7f7acc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 0x7f7acc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19f890 con 0x7f7acc102720 2024-02-26T06:23:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19f9d0 con 0x7f7acc107980 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ad18f1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19fb90 con 0x7f7acc105050 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47078/0 (socket says 172.21.15.167:47078) 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 -- 172.21.15.167:0/3502038352 learned_addr learned my addr 172.21.15.167:0/3502038352 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7acb7fe700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 0x7f7acc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 -- 172.21.15.167:0/3502038352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 msgr2=0x7f7acc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 0x7f7acc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 -- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7acc102720 msgr2=0x7f7acc197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7acc102720 0x7f7acc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 -- 172.21.15.167:0/3502038352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40095f0 con 0x7f7acc105050 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7acb7fe700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 0x7f7acc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.842+0000 7f7ac27fc700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc198280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4000c00 tx=0x7f7ab400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.843+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab401d070 con 0x7f7acc105050 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.843+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ab400fc40 con 0x7f7acc105050 2024-02-26T06:23:51.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.843+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc19fe10 con 0x7f7acc105050 2024-02-26T06:23:51.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.843+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40176e0 con 0x7f7acc105050 2024-02-26T06:23:51.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.843+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc1a02a0 con 0x7f7acc105050 2024-02-26T06:23:51.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.844+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc0fdcb0 con 0x7f7acc105050 2024-02-26T06:23:51.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.845+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f7ab40229e0 con 0x7f7acc105050 2024-02-26T06:23:51.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.845+0000 7f7ad08ef700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f7ab80a21d0 0x7f7ab80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:23:51.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.846+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f7ab40cef40 con 0x7f7acc105050 2024-02-26T06:23:51.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.846+0000 7f7acaffd700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f7ab80a21d0 0x7f7ab80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:23:51.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.847+0000 7f7acaffd700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f7ab80a21d0 0x7f7ab80a4690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7acc199220 tx=0x7f7abc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:23:51.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:51.849+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ab40985d0 con 0x7f7acc105050 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi090 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi090 to 4270M 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:23:51.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:23:52.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:23:52.054+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7acc02d960 con 0x7f7ab80a21d0 2024-02-26T06:23:52.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:52 smithi167 ceph-mon[36229]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T06:23:52.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:23:52.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:23:52.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:53.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:52 smithi006 ceph-mon[31790]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T06:23:53.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:23:53.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:23:53.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:52 smithi090 ceph-mon[35681]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T06:23:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T06:23:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T06:23:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:23:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:53 smithi167 ceph-mon[36229]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:53 smithi167 ceph-mon[36229]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:53 smithi167 ceph-mon[36229]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T06:23:54.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:53 smithi006 ceph-mon[31790]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:54.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:53 smithi006 ceph-mon[31790]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:54.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:53 smithi006 ceph-mon[31790]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T06:23:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:53 smithi090 ceph-mon[35681]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:23:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:53 smithi090 ceph-mon[35681]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:53 smithi090 ceph-mon[35681]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T06:23:55.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:54 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/2933715082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]: dispatch 2024-02-26T06:23:55.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:54 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]: dispatch 2024-02-26T06:23:55.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:54 smithi167 ceph-mon[36229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]': finished 2024-02-26T06:23:55.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:54 smithi167 ceph-mon[36229]: osdmap e77: 12 total, 11 up, 12 in 2024-02-26T06:23:55.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:23:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:54 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/2933715082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]: dispatch 2024-02-26T06:23:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:54 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]: dispatch 2024-02-26T06:23:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:54 smithi006 ceph-mon[31790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]': finished 2024-02-26T06:23:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:54 smithi006 ceph-mon[31790]: osdmap e77: 12 total, 11 up, 12 in 2024-02-26T06:23:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:23:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:54 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/2933715082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]: dispatch 2024-02-26T06:23:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:54 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]: dispatch 2024-02-26T06:23:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:54 smithi090 ceph-mon[35681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f631c7a6-a02d-434a-9f64-3b60310f0222"}]': finished 2024-02-26T06:23:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:54 smithi090 ceph-mon[35681]: osdmap e77: 12 total, 11 up, 12 in 2024-02-26T06:23:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:23:56.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:55 smithi167 ceph-mon[36229]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:56.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:55 smithi167 ceph-mon[36229]: from='client.? 172.21.15.167:0/297941902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:56.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:55 smithi006 ceph-mon[31790]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:56.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:55 smithi006 ceph-mon[31790]: from='client.? 172.21.15.167:0/297941902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:55 smithi090 ceph-mon[35681]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:23:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:55 smithi090 ceph-mon[35681]: from='client.? 172.21.15.167:0/297941902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T06:23:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:57 smithi006 ceph-mon[31790]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-02-26T06:23:58.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:57 smithi167 ceph-mon[36229]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-02-26T06:23:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:57 smithi090 ceph-mon[35681]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-02-26T06:24:00.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:23:59 smithi006 ceph-mon[31790]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-02-26T06:24:00.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:23:59 smithi167 ceph-mon[36229]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-02-26T06:24:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:23:59 smithi090 ceph-mon[35681]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-02-26T06:24:02.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:01 smithi167 ceph-mon[36229]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:24:02.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:01 smithi006 ceph-mon[31790]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:24:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:01 smithi090 ceph-mon[35681]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T06:24:03.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T06:24:03.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:02 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:03.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T06:24:03.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:02 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T06:24:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:02 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:03 smithi006 ceph-mon[31790]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-26T06:24:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:03 smithi006 ceph-mon[31790]: Deploying daemon osd.11 on smithi167 2024-02-26T06:24:04.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:03 smithi167 ceph-mon[36229]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-26T06:24:04.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:03 smithi167 ceph-mon[36229]: Deploying daemon osd.11 on smithi167 2024-02-26T06:24:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:03 smithi090 ceph-mon[35681]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-26T06:24:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:03 smithi090 ceph-mon[35681]: Deploying daemon osd.11 on smithi167 2024-02-26T06:24:06.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:05 smithi006 ceph-mon[31790]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:24:06.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:05 smithi167 ceph-mon[36229]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:24:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:05 smithi090 ceph-mon[35681]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T06:24:08.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:07 smithi006 ceph-mon[31790]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-26T06:24:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:07 smithi090 ceph-mon[35681]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-26T06:24:08.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:07 smithi167 ceph-mon[36229]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-26T06:24:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:09 smithi167 ceph-mon[36229]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:09.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:09 smithi090 ceph-mon[35681]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:09.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:09.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:09.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:09.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:09 smithi006 ceph-mon[31790]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:09.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:09.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:09.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:11.376 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:11 smithi006 ceph-mon[31790]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:11.376 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.376 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.376 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:11.376 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:11.376 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:11 smithi167 ceph-mon[36229]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:11.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:11.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:11.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:11 smithi090 ceph-mon[35681]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:11.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.995+0000 7f7ad08ef700 1 -- 172.21.15.167:0/3502038352 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7acc02d960 con 0x7f7ab80a21d0 2024-02-26T06:24:11.995 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-02-26T06:24:11.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f7ab80a21d0 msgr2=0x7f7ab80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:11.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f7ab80a21d0 0x7f7ab80a4690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7acc199220 tx=0x7f7abc006d20 comp rx=0 tx=0).stop 2024-02-26T06:24:11.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 msgr2=0x7f7acc198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:11.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc198280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4000c00 tx=0x7f7ab400f690 comp rx=0 tx=0).stop 2024-02-26T06:24:11.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 shutdown_connections 2024-02-26T06:24:11.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f7ab80a21d0 0x7f7ab80a4690 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:11.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7acc107980 0x7f7acc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:11.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7acc102720 0x7f7acc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:11.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 --2- 172.21.15.167:0/3502038352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7acc105050 0x7f7acc198280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:11.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 >> 172.21.15.167:0/3502038352 conn(0x7f7acc0fc130 msgr2=0x7f7acc102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:11.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 shutdown_connections 2024-02-26T06:24:11.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:11.997+0000 7f7ad18f1700 1 -- 172.21.15.167:0/3502038352 wait complete. 2024-02-26T06:24:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:12.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:12.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:12.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:12 smithi167 ceph-mon[36229]: from='osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:24:12.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:12 smithi167 ceph-mon[36229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:24:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:12.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:12 smithi090 ceph-mon[35681]: from='osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:24:12.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:12 smithi090 ceph-mon[35681]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:24:12.618 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.11.service 2024-02-26T06:24:12.620 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-26T06:24:12.621 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd stat -f json 2024-02-26T06:24:12.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:12.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:12.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:12.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:12 smithi006 ceph-mon[31790]: from='osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:24:12.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:12 smithi006 ceph-mon[31790]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T06:24:13.089 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:24:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:13 smithi167 ceph-mon[36229]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:13 smithi167 ceph-mon[36229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T06:24:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:13 smithi167 ceph-mon[36229]: from='osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:24:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:13 smithi167 ceph-mon[36229]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T06:24:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:13 smithi167 ceph-mon[36229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:24:14.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:13 smithi006 ceph-mon[31790]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:14.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:13 smithi006 ceph-mon[31790]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T06:24:14.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:13 smithi006 ceph-mon[31790]: from='osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:24:14.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:13 smithi006 ceph-mon[31790]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T06:24:14.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:14.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:13 smithi006 ceph-mon[31790]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:24:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:13 smithi090 ceph-mon[35681]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:13 smithi090 ceph-mon[35681]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T06:24:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:13 smithi090 ceph-mon[35681]: from='osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:24:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:13 smithi090 ceph-mon[35681]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T06:24:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:13 smithi090 ceph-mon[35681]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-02-26T06:24:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:14 smithi006 ceph-mon[31790]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:24:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:14 smithi006 ceph-mon[31790]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T06:24:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:14 smithi006 ceph-mon[31790]: from='osd.11 ' entity='osd.11' 2024-02-26T06:24:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-mon[36229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:24:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-mon[36229]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T06:24:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-mon[36229]: from='osd.11 ' entity='osd.11' 2024-02-26T06:24:15.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:24:14.847+0000 7fd7b0144700 -1 osd.11 0 waiting for initial osdmap 2024-02-26T06:24:15.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:24:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:24:14.868+0000 7fd7a8730700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T06:24:15.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:14 smithi090 ceph-mon[35681]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-02-26T06:24:15.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:14 smithi090 ceph-mon[35681]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T06:24:15.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:15.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:15.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:14 smithi090 ceph-mon[35681]: from='osd.11 ' entity='osd.11' 2024-02-26T06:24:15.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 -- 172.21.15.6:0/2143993173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18107990 msgr2=0x7f9f18109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 --2- 172.21.15.6:0/2143993173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18107990 0x7f9f18109e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009f90 tx=0x7f9f1400b330 comp rx=0 tx=0).stop 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 -- 172.21.15.6:0/2143993173 shutdown_connections 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 --2- 172.21.15.6:0/2143993173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18107990 0x7f9f18109e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 --2- 172.21.15.6:0/2143993173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f18102730 0x7f9f18104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 --2- 172.21.15.6:0/2143993173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18105060 0x7f9f18107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 -- 172.21.15.6:0/2143993173 >> 172.21.15.6:0/2143993173 conn(0x7f9f180fc140 msgr2=0x7f9f180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 -- 172.21.15.6:0/2143993173 shutdown_connections 2024-02-26T06:24:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.371+0000 7f9f20800700 1 -- 172.21.15.6:0/2143993173 wait complete. 2024-02-26T06:24:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.373+0000 7f9f20800700 1 Processor -- start 2024-02-26T06:24:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.373+0000 7f9f20800700 1 -- start start 2024-02-26T06:24:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f20800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 0x7f9f1819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:15.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f20800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18105060 0x7f9f1819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:15.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f20800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f18107990 0x7f9f181a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:15.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f20800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181a1bb0 con 0x7f9f18107990 2024-02-26T06:24:15.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f20800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181a1cf0 con 0x7f9f18105060 2024-02-26T06:24:15.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f20800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181a1e80 con 0x7f9f18102730 2024-02-26T06:24:15.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f1de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 0x7f9f1819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:15.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f1d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18105060 0x7f9f1819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f1de00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 0x7f9f1819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:40892/0 (socket says 172.21.15.6:40892) 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f1de00700 1 -- 172.21.15.6:0/2038652160 learned_addr learned my addr 172.21.15.6:0/2038652160 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.374+0000 7f9f1e800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f18107990 0x7f9f181a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1de00700 1 -- 172.21.15.6:0/2038652160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18105060 msgr2=0x7f9f1819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1de00700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18105060 0x7f9f1819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1de00700 1 -- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f18107990 msgr2=0x7f9f181a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1de00700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f18107990 0x7f9f181a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1de00700 1 -- 172.21.15.6:0/2038652160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f080095f0 con 0x7f9f18102730 2024-02-26T06:24:15.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1d400700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18105060 0x7f9f1819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.375+0000 7f9f1de00700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 0x7f9f1819a020 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e9d0 tx=0x7f9f0c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.376+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c009790 con 0x7f9f18102730 2024-02-26T06:24:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.376+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f0c00ee70 con 0x7f9f18102730 2024-02-26T06:24:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.376+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c018680 con 0x7f9f18102730 2024-02-26T06:24:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.376+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f14009c40 con 0x7f9f18102730 2024-02-26T06:24:15.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.376+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f181a2480 con 0x7f9f18102730 2024-02-26T06:24:15.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.377+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f18109510 con 0x7f9f18102730 2024-02-26T06:24:15.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.379+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9f0c00cbd0 con 0x7f9f18102730 2024-02-26T06:24:15.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.381+0000 7f9f12a00700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9f000a21d0 0x7f9f000a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:15.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.381+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f9f0c014070 con 0x7f9f18102730 2024-02-26T06:24:15.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.381+0000 7f9f1d400700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9f000a21d0 0x7f9f000a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:15.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.382+0000 7f9f1d400700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9f000a21d0 0x7f9f000a4690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9f08005fd0 tx=0x7f9f08009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:15.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.382+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f0c096d40 con 0x7f9f18102730 2024-02-26T06:24:15.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.612+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9f1810d620 con 0x7f9f18102730 2024-02-26T06:24:15.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.618+0000 7f9f12a00700 1 -- 172.21.15.6:0/2038652160 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f9f0c09a400 con 0x7f9f18102730 2024-02-26T06:24:15.618 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:15.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9f000a21d0 msgr2=0x7f9f000a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:15.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9f000a21d0 0x7f9f000a4690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9f08005fd0 tx=0x7f9f08009310 comp rx=0 tx=0).stop 2024-02-26T06:24:15.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 msgr2=0x7f9f1819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:15.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 0x7f9f1819a020 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e9d0 tx=0x7f9f0c00ece0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 shutdown_connections 2024-02-26T06:24:15.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9f000a21d0 0x7f9f000a4690 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f18107990 0x7f9f181a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f18102730 0x7f9f1819a020 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 --2- 172.21.15.6:0/2038652160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f18105060 0x7f9f1819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:15.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.621+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 >> 172.21.15.6:0/2038652160 conn(0x7f9f180fc140 msgr2=0x7f9f18101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:15.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.622+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 shutdown_connections 2024-02-26T06:24:15.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:15.622+0000 7f9f20800700 1 -- 172.21.15.6:0/2038652160 wait complete. 2024-02-26T06:24:16.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: purged_snaps scrub starts 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: purged_snaps scrub ok 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2038652160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] boot 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: osdmap e80: 12 total, 12 up, 12 in 2024-02-26T06:24:16.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:15 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:16.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: purged_snaps scrub starts 2024-02-26T06:24:16.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: purged_snaps scrub ok 2024-02-26T06:24:16.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:16.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:16.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2038652160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] boot 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: osdmap e80: 12 total, 12 up, 12 in 2024-02-26T06:24:16.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:15 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:16.382 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: purged_snaps scrub starts 2024-02-26T06:24:16.382 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: purged_snaps scrub ok 2024-02-26T06:24:16.382 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-26T06:24:16.382 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2038652160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:16.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:16.384 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:16.384 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: osd.11 [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] boot 2024-02-26T06:24:16.384 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: osdmap e80: 12 total, 12 up, 12 in 2024-02-26T06:24:16.384 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:15 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:24:16.439 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1708928629,"num_in_osds":12,"osd_in_since":1708928634,"num_remapped_pgs":0} 2024-02-26T06:24:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:17 smithi090 ceph-mon[35681]: Detected new or changed devices on smithi167 2024-02-26T06:24:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:17 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi167 to 4270M 2024-02-26T06:24:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:17 smithi006 ceph-mon[31790]: Detected new or changed devices on smithi167 2024-02-26T06:24:17.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:17 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi167 to 4270M 2024-02-26T06:24:17.440 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd stat -f json 2024-02-26T06:24:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:17 smithi167 ceph-mon[36229]: Detected new or changed devices on smithi167 2024-02-26T06:24:17.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:17 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi167 to 4270M 2024-02-26T06:24:17.916 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:24:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:18 smithi090 ceph-mon[35681]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:18 smithi090 ceph-mon[35681]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T06:24:18.378 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:18 smithi006 ceph-mon[31790]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:18.378 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:18 smithi006 ceph-mon[31790]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T06:24:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:18 smithi167 ceph-mon[36229]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:18 smithi167 ceph-mon[36229]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T06:24:19.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:19 smithi090 ceph-mon[35681]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:19.240 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:19 smithi006 ceph-mon[31790]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:19.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:19 smithi167 ceph-mon[36229]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:20.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.047+0000 7f0f28400700 1 -- 172.21.15.6:0/813405183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20103840 msgr2=0x7f0f2010a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.047+0000 7f0f28400700 1 --2- 172.21.15.6:0/813405183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20103840 0x7f0f2010a0f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800b0d0 tx=0x7f0f1800b3e0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 -- 172.21.15.6:0/813405183 shutdown_connections 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 --2- 172.21.15.6:0/813405183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20103840 0x7f0f2010a0f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 --2- 172.21.15.6:0/813405183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20101ca0 0x7f0f201020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 --2- 172.21.15.6:0/813405183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f20103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 -- 172.21.15.6:0/813405183 >> 172.21.15.6:0/813405183 conn(0x7f0f200fd280 msgr2=0x7f0f200ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 -- 172.21.15.6:0/813405183 shutdown_connections 2024-02-26T06:24:20.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.048+0000 7f0f28400700 1 -- 172.21.15.6:0/813405183 wait complete. 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.049+0000 7f0f28400700 1 Processor -- start 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.049+0000 7f0f28400700 1 -- start start 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f28400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 0x7f0f20199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f28400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f2019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f28400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103840 0x7f0f201a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f28400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201a1b20 con 0x7f0f20101ca0 2024-02-26T06:24:20.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f28400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201a1c60 con 0x7f0f20103840 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f28400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201a1df0 con 0x7f0f20102ea0 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f25a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 0x7f0f20199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f25000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f2019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f25a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 0x7f0f20199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41974/0 (socket says 172.21.15.6:41974) 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f25a00700 1 -- 172.21.15.6:0/1245140586 learned_addr learned my addr 172.21.15.6:0/1245140586 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f25000700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f2019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:50226/0 (socket says 172.21.15.6:50226) 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.050+0000 7f0f26400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103840 0x7f0f201a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25a00700 1 -- 172.21.15.6:0/1245140586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 msgr2=0x7f0f2019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25a00700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f2019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25a00700 1 -- 172.21.15.6:0/1245140586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103840 msgr2=0x7f0f201a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:20.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25a00700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103840 0x7f0f201a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25a00700 1 -- 172.21.15.6:0/1245140586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f18009d00 con 0x7f0f20101ca0 2024-02-26T06:24:20.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25000700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f2019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:20.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f25a00700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 0x7f0f20199f90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400d9b0 tx=0x7f0f1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:20.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14004020 con 0x7f0f20101ca0 2024-02-26T06:24:20.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f14009730 con 0x7f0f20101ca0 2024-02-26T06:24:20.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14003b40 con 0x7f0f20101ca0 2024-02-26T06:24:20.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.051+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f201a20d0 con 0x7f0f20101ca0 2024-02-26T06:24:20.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.052+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f201a2580 con 0x7f0f20101ca0 2024-02-26T06:24:20.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.053+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f20068930 con 0x7f0f20101ca0 2024-02-26T06:24:20.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.055+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0f14009ba0 con 0x7f0f20101ca0 2024-02-26T06:24:20.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.057+0000 7f0f1e000700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0f080a21d0 0x7f0f080a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:20.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.058+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f140cd750 con 0x7f0f20101ca0 2024-02-26T06:24:20.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.058+0000 7f0f25000700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0f080a21d0 0x7f0f080a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:20.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.058+0000 7f0f25000700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0f080a21d0 0x7f0f080a4690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005950 tx=0x7f0f0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:20.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.061+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f14096ba0 con 0x7f0f20101ca0 2024-02-26T06:24:20.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.288+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0f2010d8f0 con 0x7f0f20101ca0 2024-02-26T06:24:20.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.289+0000 7f0f1e000700 1 -- 172.21.15.6:0/1245140586 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0f1409a260 con 0x7f0f20101ca0 2024-02-26T06:24:20.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0f080a21d0 msgr2=0x7f0f080a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0f080a21d0 0x7f0f080a4690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005950 tx=0x7f0f0c00b410 comp rx=0 tx=0).stop 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 msgr2=0x7f0f20199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 0x7f0f20199f90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400d9b0 tx=0x7f0f1400dcc0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 shutdown_connections 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0f080a21d0 0x7f0f080a4690 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f20103840 0x7f0f201a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f20101ca0 0x7f0f20199f90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 --2- 172.21.15.6:0/1245140586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f20102ea0 0x7f0f2019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 >> 172.21.15.6:0/1245140586 conn(0x7f0f200fd280 msgr2=0x7f0f20073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 shutdown_connections 2024-02-26T06:24:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:20.292+0000 7f0f28400700 1 -- 172.21.15.6:0/1245140586 wait complete. 2024-02-26T06:24:21.117 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1708928655,"num_in_osds":12,"osd_in_since":1708928634,"num_remapped_pgs":0} 2024-02-26T06:24:21.118 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T06:24:21.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:21 smithi006 ceph-mon[31790]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:21.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:21 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1245140586' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:24:21.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:21 smithi167 ceph-mon[36229]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:21.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:21 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1245140586' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:24:21.597 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:24:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:21 smithi090 ceph-mon[35681]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:21 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1245140586' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T06:24:23.391 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:23 smithi006 ceph-mon[31790]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:23.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:23 smithi167 ceph-mon[36229]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:23 smithi090 ceph-mon[35681]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:23.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 -- 172.21.15.6:0/2112084098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 msgr2=0x7f0298115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:23.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 --2- 172.21.15.6:0/2112084098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f0298115060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0290009990 tx=0x7f0290009ca0 comp rx=0 tx=0).stop 2024-02-26T06:24:23.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 -- 172.21.15.6:0/2112084098 shutdown_connections 2024-02-26T06:24:23.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 --2- 172.21.15.6:0/2112084098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f0298115060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:23.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 --2- 172.21.15.6:0/2112084098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0298108400 0x7f02981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:23.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 --2- 172.21.15.6:0/2112084098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029810cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:23.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 -- 172.21.15.6:0/2112084098 >> 172.21.15.6:0/2112084098 conn(0x7f0298077910 msgr2=0x7f0298079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:23.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 -- 172.21.15.6:0/2112084098 shutdown_connections 2024-02-26T06:24:23.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.870+0000 7f0297200700 1 -- 172.21.15.6:0/2112084098 wait complete. 2024-02-26T06:24:23.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.871+0000 7f0297200700 1 Processor -- start 2024-02-26T06:24:23.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 -- start start 2024-02-26T06:24:23.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0298108400 0x7f029819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:23.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:23.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f02981a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:23.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02981a34c0 con 0x7f0298108db0 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02981a3630 con 0x7f0298108400 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0297200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02981a37f0 con 0x7f029810d490 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f028d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0296800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f02981a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f028d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41980/0 (socket says 172.21.15.6:41980) 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f028d400700 1 -- 172.21.15.6:0/1801017662 learned_addr learned my addr 172.21.15.6:0/1801017662 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.872+0000 7f0295e00700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0298108400 0x7f029819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f0296800700 1 -- 172.21.15.6:0/1801017662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0298108400 msgr2=0x7f029819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f0296800700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0298108400 0x7f029819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f0296800700 1 -- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 msgr2=0x7f029819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f0296800700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:23.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f0296800700 1 -- 172.21.15.6:0/1801017662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900095f0 con 0x7f029810d490 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f028d400700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f0296800700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f02981a2eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02900059d0 tx=0x7f0290005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.873+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029001d070 con 0x7f029810d490 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.874+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02981a3a70 con 0x7f029810d490 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.874+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f029000b610 con 0x7f029810d490 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.874+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290021dc0 con 0x7f029810d490 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.874+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02981ab330 con 0x7f029810d490 2024-02-26T06:24:23.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.875+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0298068930 con 0x7f029810d490 2024-02-26T06:24:23.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.877+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0290021410 con 0x7f029810d490 2024-02-26T06:24:23.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.878+0000 7f028f400700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f027c0a2150 0x7f027c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:23.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.878+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02900ceab0 con 0x7f029810d490 2024-02-26T06:24:23.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.878+0000 7f0295e00700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f027c0a2150 0x7f027c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:23.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.879+0000 7f0295e00700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f027c0a2150 0x7f027c0a4610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0288009c60 tx=0x7f0288009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:23.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:23.880+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0290097f10 con 0x7f029810d490 2024-02-26T06:24:24.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.104+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02981181f0 con 0x7f029810d490 2024-02-26T06:24:24.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.106+0000 7f028f400700 1 -- 172.21.15.6:0/1801017662 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f029009b5d0 con 0x7f029810d490 2024-02-26T06:24:24.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":81,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:24:16.991212+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:24:24.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f027c0a2150 msgr2=0x7f027c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:24.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f027c0a2150 0x7f027c0a4610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0288009c60 tx=0x7f0288009260 comp rx=0 tx=0).stop 2024-02-26T06:24:24.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 msgr2=0x7f02981a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:24.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f02981a2eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02900059d0 tx=0x7f0290005720 comp rx=0 tx=0).stop 2024-02-26T06:24:24.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 shutdown_connections 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f027c0a2150 0x7f027c0a4610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f029810d490 0x7f02981a2eb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0298108400 0x7f029819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 --2- 172.21.15.6:0/1801017662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0298108db0 0x7f029819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 >> 172.21.15.6:0/1801017662 conn(0x7f0298077910 msgr2=0x7f0298078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 shutdown_connections 2024-02-26T06:24:24.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:24.109+0000 7f0297200700 1 -- 172.21.15.6:0/1801017662 wait complete. 2024-02-26T06:24:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:24 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1801017662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:24:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:24 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1801017662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:24:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:24 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1801017662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:24:25.019 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:24:25.020 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-26T06:24:25.412 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:25 smithi167 ceph-mon[36229]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:25.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:25 smithi006 ceph-mon[31790]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:25.505 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:24:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:25 smithi090 ceph-mon[35681]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:27.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:27 smithi006 ceph-mon[31790]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:27.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:27 smithi167 ceph-mon[36229]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:27 smithi090 ceph-mon[35681]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:27.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 -- 172.21.15.6:0/2360131107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 msgr2=0x7f061810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:27.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 --2- 172.21.15.6:0/2360131107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 0x7f061810eac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f061000b0a0 tx=0x7f061000b3b0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 -- 172.21.15.6:0/2360131107 shutdown_connections 2024-02-26T06:24:27.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 --2- 172.21.15.6:0/2360131107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 0x7f061810eac0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 --2- 172.21.15.6:0/2360131107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618107ca0 0x7f0618108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 --2- 172.21.15.6:0/2360131107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618101cc0 0x7f0618102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 -- 172.21.15.6:0/2360131107 >> 172.21.15.6:0/2360131107 conn(0x7f0618078da0 msgr2=0x7f06180791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:27.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 -- 172.21.15.6:0/2360131107 shutdown_connections 2024-02-26T06:24:27.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.664+0000 7f061d600700 1 -- 172.21.15.6:0/2360131107 wait complete. 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.665+0000 7f061d600700 1 Processor -- start 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 -- start start 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 0x7f061819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 0x7f061819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618107ca0 0x7f061819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061819f360 con 0x7f0618107ca0 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061819f4d0 con 0x7f0618101cc0 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f061d600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061819f690 con 0x7f0618102680 2024-02-26T06:24:27.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f0617400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618107ca0 0x7f061819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f0616a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 0x7f061819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f0616a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 0x7f061819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34930/0 (socket says 172.21.15.6:34930) 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.666+0000 7f0616a00700 1 -- 172.21.15.6:0/1310633619 learned_addr learned my addr 172.21.15.6:0/1310633619 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0616000700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 0x7f061819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0617400700 1 -- 172.21.15.6:0/1310633619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 msgr2=0x7f061819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0617400700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 0x7f061819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0617400700 1 -- 172.21.15.6:0/1310633619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 msgr2=0x7f061819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0617400700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 0x7f061819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0617400700 1 -- 172.21.15.6:0/1310633619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608009520 con 0x7f0618107ca0 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0616a00700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 0x7f061819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.667+0000 7f0617400700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618107ca0 0x7f061819ed50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f061000b070 tx=0x7f061000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:27.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.668+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000e040 con 0x7f0618107ca0 2024-02-26T06:24:27.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.668+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f061001d470 con 0x7f0618107ca0 2024-02-26T06:24:27.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.668+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0610009d00 con 0x7f0618107ca0 2024-02-26T06:24:27.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.668+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100052a0 con 0x7f0618107ca0 2024-02-26T06:24:27.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.668+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06181a7000 con 0x7f0618107ca0 2024-02-26T06:24:27.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.669+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0618103350 con 0x7f0618107ca0 2024-02-26T06:24:27.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.671+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f061001d5e0 con 0x7f0618107ca0 2024-02-26T06:24:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.672+0000 7f060f400700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f05fc0a2150 0x7f05fc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.673+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06100cea50 con 0x7f0618107ca0 2024-02-26T06:24:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.673+0000 7f0616a00700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f05fc0a2150 0x7f05fc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.674+0000 7f0616a00700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f05fc0a2150 0x7f05fc0a4610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0608009fd0 tx=0x7f0608009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:27.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.677+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0610097eb0 con 0x7f0618107ca0 2024-02-26T06:24:27.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.898+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0618111d00 con 0x7f0618107ca0 2024-02-26T06:24:27.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.899+0000 7f060f400700 1 -- 172.21.15.6:0/1310633619 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f061009b570 con 0x7f0618107ca0 2024-02-26T06:24:27.900 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:24:27.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.902+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f05fc0a2150 msgr2=0x7f05fc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:27.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.902+0000 7f061d600700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f05fc0a2150 0x7f05fc0a4610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0608009fd0 tx=0x7f0608009260 comp rx=0 tx=0).stop 2024-02-26T06:24:27.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.902+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618107ca0 msgr2=0x7f061819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:27.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.902+0000 7f061d600700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618107ca0 0x7f061819ed50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f061000b070 tx=0x7f061000be20 comp rx=0 tx=0).stop 2024-02-26T06:24:27.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.902+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 shutdown_connections 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f05fc0a2150 0x7f05fc0a4610 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0618107ca0 0x7f061819ed50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0618101cc0 0x7f061819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 --2- 172.21.15.6:0/1310633619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0618102680 0x7f061819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 >> 172.21.15.6:0/1310633619 conn(0x7f0618078da0 msgr2=0x7f061810cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 shutdown_connections 2024-02-26T06:24:27.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:27.903+0000 7f061d600700 1 -- 172.21.15.6:0/1310633619 wait complete. 2024-02-26T06:24:28.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:28 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1310633619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:24:28.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:28 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1310633619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:24:28.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:28 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1310633619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.a on smithi006 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.d on smithi006 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.g on smithi006 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.j on smithi006 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.b on smithi090 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.e on smithi090 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.h on smithi090 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.k on smithi090 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-02-26T06:24:28.889 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-02-26T06:24:28.890 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-02-26T06:24:28.890 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-02-26T06:24:28.890 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph orch apply mds all '12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-02-26T06:24:29.143 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:24:29.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:29 smithi167 ceph-mon[36229]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:29.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:29 smithi090 ceph-mon[35681]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:29.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:29 smithi006 ceph-mon[31790]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/178786298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94101ca0 msgr2=0x7fbe941020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/178786298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94101ca0 0x7fbe941020c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009990 tx=0x7fbe88009ca0 comp rx=0 tx=0).stop 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/178786298 shutdown_connections 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/178786298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe94103840 0x7fbe9410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/178786298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94101ca0 0x7fbe941020c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/178786298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe94102ea0 0x7fbe94103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/178786298 >> 172.21.15.167:0/178786298 conn(0x7fbe940fd280 msgr2=0x7fbe940ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/178786298 shutdown_connections 2024-02-26T06:24:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/178786298 wait complete. 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 Processor -- start 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- start start 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe94101ca0 0x7fbe94075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe94102ea0 0x7fbe94074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 0x7fbe940745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94074d20 con 0x7fbe94101ca0 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94074e90 con 0x7fbe94102ea0 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.806+0000 7fbe9b1f6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a1ca0 con 0x7fbe94103840 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 0x7fbe940745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 0x7fbe940745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55178/0 (socket says 172.21.15.167:55178) 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 -- 172.21.15.167:0/3706283445 learned_addr learned my addr 172.21.15.167:0/3706283445 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe93fff700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe94102ea0 0x7fbe94074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 -- 172.21.15.167:0/3706283445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe94102ea0 msgr2=0x7fbe94074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe94102ea0 0x7fbe94074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 -- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe94101ca0 msgr2=0x7fbe94075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe94101ca0 0x7fbe94075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 -- 172.21.15.167:0/3706283445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880095f0 con 0x7fbe94103840 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe99793700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 0x7fbe940745a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00b970 tx=0x7fbe8c00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00c9b0 con 0x7fbe94103840 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c014440 con 0x7fbe94103840 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe941a1f80 con 0x7fbe94103840 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0075d0 con 0x7fbe94103840 2024-02-26T06:24:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.807+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941a2420 con 0x7fbe94103840 2024-02-26T06:24:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.808+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe941090e0 con 0x7fbe94103840 2024-02-26T06:24:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.809+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fbe8c014920 con 0x7fbe94103840 2024-02-26T06:24:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.810+0000 7fbe91ffb700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbe840a2150 0x7fbe840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.810+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe8c006e40 con 0x7fbe94103840 2024-02-26T06:24:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.810+0000 7fbe98f92700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbe840a2150 0x7fbe840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:30.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.811+0000 7fbe98f92700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbe840a2150 0x7fbe840a4610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbe880095c0 tx=0x7fbe88009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:30.814+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe8c0963f0 con 0x7fbe94103840 2024-02-26T06:24:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.020+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe9402d970 con 0x7fbe840a2150 2024-02-26T06:24:31.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.037+0000 7fbe91ffb700 1 -- 172.21.15.167:0/3706283445 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fbe9402d970 con 0x7fbe840a2150 2024-02-26T06:24:31.037 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-02-26T06:24:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbe840a2150 msgr2=0x7fbe840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbe840a2150 0x7fbe840a4610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbe880095c0 tx=0x7fbe88009510 comp rx=0 tx=0).stop 2024-02-26T06:24:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 msgr2=0x7fbe940745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 0x7fbe940745a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00b970 tx=0x7fbe8c00bd30 comp rx=0 tx=0).stop 2024-02-26T06:24:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 shutdown_connections 2024-02-26T06:24:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbe840a2150 0x7fbe840a4610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe94103840 0x7fbe940745a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe94101ca0 0x7fbe94075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 --2- 172.21.15.167:0/3706283445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe94102ea0 0x7fbe94074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 >> 172.21.15.167:0/3706283445 conn(0x7fbe940fd280 msgr2=0x7fbe94106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 shutdown_connections 2024-02-26T06:24:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-02-26T06:24:31.039+0000 7fbe9b1f6700 1 -- 172.21.15.167:0/3706283445 wait complete. 2024-02-26T06:24:31.348 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:31 smithi167 ceph-mon[36229]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:31.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:31.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:31 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:31 smithi090 ceph-mon[35681]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:31 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:31 smithi006 ceph-mon[31790]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:31 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:24:31.678 DEBUG:teuthology.orchestra.run.smithi006:mds.a> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.a.service 2024-02-26T06:24:31.686 DEBUG:teuthology.orchestra.run.smithi006:mds.d> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.d.service 2024-02-26T06:24:31.731 DEBUG:teuthology.orchestra.run.smithi006:mds.g> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.g.service 2024-02-26T06:24:31.779 DEBUG:teuthology.orchestra.run.smithi006:mds.j> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.j.service 2024-02-26T06:24:31.824 DEBUG:teuthology.orchestra.run.smithi090:mds.b> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.b.service 2024-02-26T06:24:31.826 DEBUG:teuthology.orchestra.run.smithi090:mds.e> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.e.service 2024-02-26T06:24:31.869 DEBUG:teuthology.orchestra.run.smithi090:mds.h> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.h.service 2024-02-26T06:24:31.912 DEBUG:teuthology.orchestra.run.smithi090:mds.k> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.k.service 2024-02-26T06:24:31.956 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.c.service 2024-02-26T06:24:31.958 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.f.service 2024-02-26T06:24:32.003 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.i.service 2024-02-26T06:24:32.046 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.l.service 2024-02-26T06:24:32.089 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-26T06:24:32.090 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:24:32.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:32 smithi167 ceph-mon[36229]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:24:32.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:32 smithi167 ceph-mon[36229]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 -- 172.21.15.6:0/256768553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8106f90 msgr2=0x7f42b8109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 --2- 172.21.15.6:0/256768553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8106f90 0x7f42b8109380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009900 tx=0x7f42b002bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 -- 172.21.15.6:0/256768553 shutdown_connections 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 --2- 172.21.15.6:0/256768553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8106f90 0x7f42b8109380 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 --2- 172.21.15.6:0/256768553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8104660 0x7f42b8106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 --2- 172.21.15.6:0/256768553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 0x7f42b8104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 -- 172.21.15.6:0/256768553 >> 172.21.15.6:0/256768553 conn(0x7f42b80fb6a0 msgr2=0x7f42b80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 -- 172.21.15.6:0/256768553 shutdown_connections 2024-02-26T06:24:32.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.327+0000 7f42be800640 1 -- 172.21.15.6:0/256768553 wait complete. 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.328+0000 7f42be800640 1 Processor -- start 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 -- start start 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 0x7f42b8101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8104660 0x7f42b80ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8106f90 0x7f42b8100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81008f0 con 0x7f42b8104660 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b8100a60 con 0x7f42b8106f90 2024-02-26T06:24:32.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42be800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81a1320 con 0x7f42b8101d30 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42aea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8104660 0x7f42b80ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 0x7f42b8101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42aea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8104660 0x7f42b80ffce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53458/0 (socket says 172.21.15.6:53458) 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42aea00640 1 -- 172.21.15.6:0/3571403277 learned_addr learned my addr 172.21.15.6:0/3571403277 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.329+0000 7f42b7e00640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8106f90 0x7f42b8100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7400640 1 -- 172.21.15.6:0/3571403277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8106f90 msgr2=0x7f42b8100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7400640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8106f90 0x7f42b8100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7400640 1 -- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8104660 msgr2=0x7f42b80ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7400640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8104660 0x7f42b80ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7400640 1 -- 172.21.15.6:0/3571403277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009490 con 0x7f42b8101d30 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7e00640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8106f90 0x7f42b8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b7400640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 0x7f42b8101600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009fd0 tx=0x7f42a800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800cea0 con 0x7f42b8101d30 2024-02-26T06:24:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42a8005020 con 0x7f42b8101d30 2024-02-26T06:24:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80106d0 con 0x7f42b8101d30 2024-02-26T06:24:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.330+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b0009560 con 0x7f42b8101d30 2024-02-26T06:24:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.331+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b81a1880 con 0x7f42b8101d30 2024-02-26T06:24:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.332+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b80fd190 con 0x7f42b8101d30 2024-02-26T06:24:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.333+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f42a8005190 con 0x7f42b8101d30 2024-02-26T06:24:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.335+0000 7f42b4c00640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f429c0a2140 0x7f429c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.335+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f42a8014070 con 0x7f42b8101d30 2024-02-26T06:24:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.335+0000 7f42aea00640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f429c0a2140 0x7f429c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.336+0000 7f42aea00640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f429c0a2140 0x7f429c0a4600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42b8101370 tx=0x7f42a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.339+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42a809ae80 con 0x7f42b8101d30 2024-02-26T06:24:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:32 smithi006 ceph-mon[31790]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:24:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:32 smithi006 ceph-mon[31790]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-02-26T06:24:32.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.543+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42b802cdd0 con 0x7f42b8101d30 2024-02-26T06:24:32.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.545+0000 7f42b4c00640 1 -- 172.21.15.6:0/3571403277 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f42a809aa10 con 0x7f42b8101d30 2024-02-26T06:24:32.545 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:32.546 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":81,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:24:16.991212+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:24:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f429c0a2140 msgr2=0x7f429c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f429c0a2140 0x7f429c0a4600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42b8101370 tx=0x7f42a0005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 msgr2=0x7f42b8101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 0x7f42b8101600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009fd0 tx=0x7f42a800ef90 comp rx=0 tx=0).stop 2024-02-26T06:24:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 shutdown_connections 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f429c0a2140 0x7f429c0a4600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42b8106f90 0x7f42b8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42b8104660 0x7f42b80ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 --2- 172.21.15.6:0/3571403277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42b8101d30 0x7f42b8101600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 >> 172.21.15.6:0/3571403277 conn(0x7f42b80fb6a0 msgr2=0x7f42b8102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 shutdown_connections 2024-02-26T06:24:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.548+0000 7f42be800640 1 -- 172.21.15.6:0/3571403277 wait complete. 2024-02-26T06:24:32.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:24:32.569 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:24:32.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:32 smithi090 ceph-mon[35681]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:24:32.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:32 smithi090 ceph-mon[35681]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-02-26T06:24:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 -- 172.21.15.6:0/8249304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c100fc0 msgr2=0x7fad1c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 --2- 172.21.15.6:0/8249304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c100fc0 0x7fad1c1013e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009860 tx=0x7fad0c02bb70 comp rx=0 tx=0).stop 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 -- 172.21.15.6:0/8249304 shutdown_connections 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 --2- 172.21.15.6:0/8249304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c102b70 0x7fad1c109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 --2- 172.21.15.6:0/8249304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad1c1021b0 0x7fad1c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 --2- 172.21.15.6:0/8249304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c100fc0 0x7fad1c1013e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 -- 172.21.15.6:0/8249304 >> 172.21.15.6:0/8249304 conn(0x7fad1c0fc530 msgr2=0x7fad1c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 -- 172.21.15.6:0/8249304 shutdown_connections 2024-02-26T06:24:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.793+0000 7fad1b200640 1 -- 172.21.15.6:0/8249304 wait complete. 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.794+0000 7fad1b200640 1 Processor -- start 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.794+0000 7fad1b200640 1 -- start start 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1b200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 0x7fad1c074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c1021b0 0x7fad1c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad1c102b70 0x7fad1c073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1b200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c074210 con 0x7fad1c100fc0 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c074380 con 0x7fad1c1021b0 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c1a1100 con 0x7fad1c102b70 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad19e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 0x7fad1c074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad19e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 0x7fad1c074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53476/0 (socket says 172.21.15.6:53476) 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad19e00640 1 -- 172.21.15.6:0/3067315894 learned_addr learned my addr 172.21.15.6:0/3067315894 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad1a800640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad1c102b70 0x7fad1c073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.795+0000 7fad19400640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c1021b0 0x7fad1c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad19e00640 1 -- 172.21.15.6:0/3067315894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad1c102b70 msgr2=0x7fad1c073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad19e00640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad1c102b70 0x7fad1c073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad19e00640 1 -- 172.21.15.6:0/3067315894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c1021b0 msgr2=0x7fad1c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad19e00640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c1021b0 0x7fad1c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad19e00640 1 -- 172.21.15.6:0/3067315894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c009490 con 0x7fad1c100fc0 2024-02-26T06:24:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad19e00640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 0x7fad1c074f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009780 tx=0x7fad0c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c02c510 con 0x7fad1c100fc0 2024-02-26T06:24:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad0c02cb50 con 0x7fad1c100fc0 2024-02-26T06:24:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c1a1350 con 0x7fad1c100fc0 2024-02-26T06:24:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c003850 con 0x7fad1c100fc0 2024-02-26T06:24:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.796+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c1a16c0 con 0x7fad1c100fc0 2024-02-26T06:24:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.798+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c108360 con 0x7fad1c100fc0 2024-02-26T06:24:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.800+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fad0c03b040 con 0x7fad1c100fc0 2024-02-26T06:24:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.801+0000 7fad12a00640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fad000a20c0 0x7fad000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.801+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fad0c0ef400 con 0x7fad1c100fc0 2024-02-26T06:24:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.801+0000 7fad19400640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fad000a20c0 0x7fad000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:32.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.802+0000 7fad19400640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fad000a20c0 0x7fad000a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fad1c074ca0 tx=0x7fad08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:32.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:32.805+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad0c0b87d0 con 0x7fad1c100fc0 2024-02-26T06:24:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.007+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad1c02cdd0 con 0x7fad1c100fc0 2024-02-26T06:24:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.008+0000 7fad12a00640 1 -- 172.21.15.6:0/3067315894 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad0c0bbe90 con 0x7fad1c100fc0 2024-02-26T06:24:33.008 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:24:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.010+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fad000a20c0 msgr2=0x7fad000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.010+0000 7fad1b200640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fad000a20c0 0x7fad000a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fad1c074ca0 tx=0x7fad08009260 comp rx=0 tx=0).stop 2024-02-26T06:24:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.010+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 msgr2=0x7fad1c074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.010+0000 7fad1b200640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 0x7fad1c074f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009780 tx=0x7fad0c005d50 comp rx=0 tx=0).stop 2024-02-26T06:24:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 shutdown_connections 2024-02-26T06:24:33.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad1c102b70 0x7fad1c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fad000a20c0 0x7fad000a4580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad1c1021b0 0x7fad1c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 --2- 172.21.15.6:0/3067315894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad1c100fc0 0x7fad1c074f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 >> 172.21.15.6:0/3067315894 conn(0x7fad1c0fc530 msgr2=0x7fad1c105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 shutdown_connections 2024-02-26T06:24:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.011+0000 7fad1b200640 1 -- 172.21.15.6:0/3067315894 wait complete. 2024-02-26T06:24:33.033 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 -- 172.21.15.6:0/1011622435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104490 msgr2=0x7f2374104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 --2- 172.21.15.6:0/1011622435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104490 0x7f2374104910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f235c009900 tx=0x7f235c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 -- 172.21.15.6:0/1011622435 shutdown_connections 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 --2- 172.21.15.6:0/1011622435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2374104e50 0x7f237410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 --2- 172.21.15.6:0/1011622435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104490 0x7f2374104910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 --2- 172.21.15.6:0/1011622435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23741032a0 0x7f23741036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 -- 172.21.15.6:0/1011622435 >> 172.21.15.6:0/1011622435 conn(0x7f23740fe830 msgr2=0x7f2374100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:33.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 -- 172.21.15.6:0/1011622435 shutdown_connections 2024-02-26T06:24:33.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.260+0000 7f2379600640 1 -- 172.21.15.6:0/1011622435 wait complete. 2024-02-26T06:24:33.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.262+0000 7f2379600640 1 Processor -- start 2024-02-26T06:24:33.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.262+0000 7f2379600640 1 -- start start 2024-02-26T06:24:33.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.262+0000 7f2379600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23741032a0 0x7f2374199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:33.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.262+0000 7f2379600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2374104490 0x7f2374199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:33.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.262+0000 7f2379600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 0x7f23741a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:33.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.262+0000 7f2379600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741a0e20 con 0x7f2374104490 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2379600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741a0f90 con 0x7f23741032a0 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2379600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741a1150 con 0x7f2374104e50 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 0x7f23741a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f236b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2374104490 0x7f2374199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2373e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23741032a0 0x7f2374199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 0x7f23741a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:51610/0 (socket says 172.21.15.6:51610) 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 -- 172.21.15.6:0/2919341996 learned_addr learned my addr 172.21.15.6:0/2919341996 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 -- 172.21.15.6:0/2919341996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23741032a0 msgr2=0x7f2374199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23741032a0 0x7f2374199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 -- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2374104490 msgr2=0x7f2374199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2374104490 0x7f2374199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 -- 172.21.15.6:0/2919341996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009560 con 0x7f2374104e50 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.263+0000 7f2378c00640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 0x7f23741a08e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f236c009fd0 tx=0x7f236c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.264+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00bea0 con 0x7f2374104e50 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.264+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f236c009940 con 0x7f2374104e50 2024-02-26T06:24:33.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.264+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0106c0 con 0x7f2374104e50 2024-02-26T06:24:33.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.264+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23741a1430 con 0x7f2374104e50 2024-02-26T06:24:33.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.264+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23741a18c0 con 0x7f2374104e50 2024-02-26T06:24:33.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.265+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237410a5c0 con 0x7f2374104e50 2024-02-26T06:24:33.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.267+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f236c004240 con 0x7f2374104e50 2024-02-26T06:24:33.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.268+0000 7f2371600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f23580a2130 0x7f23580a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:33.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.268+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f236c0cd4a0 con 0x7f2374104e50 2024-02-26T06:24:33.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.268+0000 7f2373e00640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f23580a2130 0x7f23580a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:33.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.269+0000 7f2373e00640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f23580a2130 0x7f23580a45f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f23741042f0 tx=0x7f2364005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:33.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.272+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f236c096920 con 0x7f2374104e50 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: Deploying daemon mds.l on smithi167 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3571403277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:24:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:33 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3067315894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: Deploying daemon mds.l on smithi167 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3571403277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:24:33.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:33 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3067315894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:24:33.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.515+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f237410a030 con 0x7f2374104e50 2024-02-26T06:24:33.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.516+0000 7f2371600640 1 -- 172.21.15.6:0/2919341996 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f236c099fe0 con 0x7f2374104e50 2024-02-26T06:24:33.516 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:33.516 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-26T06:24:33.517 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 1 2024-02-26T06:24:33.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f23580a2130 msgr2=0x7f23580a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f23580a2130 0x7f23580a45f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f23741042f0 tx=0x7f2364005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 msgr2=0x7f23741a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 0x7f23741a08e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f236c009fd0 tx=0x7f236c00b260 comp rx=0 tx=0).stop 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 shutdown_connections 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2374104e50 0x7f23741a08e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2374104490 0x7f2374199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f23580a2130 0x7f23580a45f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 --2- 172.21.15.6:0/2919341996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23741032a0 0x7f2374199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 >> 172.21.15.6:0/2919341996 conn(0x7f23740fe830 msgr2=0x7f2374107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 shutdown_connections 2024-02-26T06:24:33.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:33.519+0000 7f2379600640 1 -- 172.21.15.6:0/2919341996 wait complete. 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: Deploying daemon mds.l on smithi167 2024-02-26T06:24:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3571403277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:24:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:33 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3067315894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:24:34.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:34 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2919341996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:34.495 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 systemd[1]: Starting Ceph mds.l for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:34.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:34 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2919341996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:34.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:34 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2919341996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:34.821 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 podman[66821]: 2024-02-26 06:24:34.448080028 +0000 UTC m=+0.017830473 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:34.821 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 podman[66821]: 2024-02-26 06:24:34.560815632 +0000 UTC m=+0.130566106 container create bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-26T06:24:34.821 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 podman[66821]: 2024-02-26 06:24:34.821845178 +0000 UTC m=+0.391595627 container init bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-26T06:24:35.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 podman[66821]: 2024-02-26 06:24:34.824688383 +0000 UTC m=+0.394438815 container start bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-26T06:24:35.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 ceph-mds[66841]: -- 172.21.15.167:0/3450822340 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560777c01a0 con 0x556077796c00 2024-02-26T06:24:35.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l[66837]: starting mds.l at 2024-02-26T06:24:35.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 bash[66821]: bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca 2024-02-26T06:24:35.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:24:34 smithi167 systemd[1]: Started Ceph mds.l for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:35.538 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:35.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 -- 172.21.15.6:0/2804340144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 msgr2=0x7f85681036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 --2- 172.21.15.6:0/2804340144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f85681036c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 -- 172.21.15.6:0/2804340144 shutdown_connections 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 --2- 172.21.15.6:0/2804340144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8568104e50 0x7f856810b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 --2- 172.21.15.6:0/2804340144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104490 0x7f8568104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 --2- 172.21.15.6:0/2804340144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f85681036c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 -- 172.21.15.6:0/2804340144 >> 172.21.15.6:0/2804340144 conn(0x7f85680fe830 msgr2=0x7f8568100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 -- 172.21.15.6:0/2804340144 shutdown_connections 2024-02-26T06:24:35.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.774+0000 7f856ea00640 1 -- 172.21.15.6:0/2804340144 wait complete. 2024-02-26T06:24:35.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.775+0000 7f856ea00640 1 Processor -- start 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.775+0000 7f856ea00640 1 -- start start 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.775+0000 7f856ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f8568199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.775+0000 7f856ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8568104490 0x7f8568199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104e50 0x7f85681a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856ea00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85681a0e00 con 0x7f85681032a0 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85681a0f70 con 0x7f8568104490 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85681a1130 con 0x7f8568104e50 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f8568199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f8568199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53524/0 (socket says 172.21.15.6:53524) 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 -- 172.21.15.6:0/2210114129 learned_addr learned my addr 172.21.15.6:0/2210114129 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8568104490 0x7f8568199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856e000640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104e50 0x7f85681a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 -- 172.21.15.6:0/2210114129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104e50 msgr2=0x7f85681a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:35.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104e50 0x7f85681a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 -- 172.21.15.6:0/2210114129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8568104490 msgr2=0x7f8568199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8568104490 0x7f8568199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856d600640 1 -- 172.21.15.6:0/2210114129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c009560 con 0x7f85681032a0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.776+0000 7f856e000640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104e50 0x7f85681a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.777+0000 7f856d600640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f8568199310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f855c000c00 tx=0x7f855c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.777+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c0043d0 con 0x7f85681032a0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.777+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f855c0052d0 con 0x7f85681032a0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.777+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c003da0 con 0x7f85681032a0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.777+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85681a13b0 con 0x7f85681032a0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.777+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85681a1910 con 0x7f85681032a0 2024-02-26T06:24:35.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.778+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856810a560 con 0x7f85681032a0 2024-02-26T06:24:35.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.780+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f855c005440 con 0x7f85681032a0 2024-02-26T06:24:35.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.782+0000 7f8566000640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85500a2130 0x7f85500a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:35.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.783+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f855c0f0050 con 0x7f85681032a0 2024-02-26T06:24:35.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.783+0000 7f856cc00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85500a2130 0x7f85500a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:35.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.783+0000 7f856cc00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85500a2130 0x7f85500a45f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f856819a820 tx=0x7f8558005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:35.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:35.787+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f855c0b9540 con 0x7f85681032a0 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6832/66780761,v1:172.21.15.167:6833/66780761] up:boot 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: fsmap 1 up:standby 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:35 smithi006 ceph-mon[31790]: Deploying daemon mds.k on smithi090 2024-02-26T06:24:35.961 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6832/66780761,v1:172.21.15.167:6833/66780761] up:boot 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: fsmap 1 up:standby 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:35.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:35 smithi090 ceph-mon[35681]: Deploying daemon mds.k on smithi090 2024-02-26T06:24:36.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.030+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f856802cda0 con 0x7f85681032a0 2024-02-26T06:24:36.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.031+0000 7f8566000640 1 -- 172.21.15.6:0/2210114129 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1322 (secure 0 0 0) 0x7f855c043020 con 0x7f85681032a0 2024-02-26T06:24:36.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:36.032 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T06:24:36.032 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 2 2024-02-26T06:24:36.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85500a2130 msgr2=0x7f85500a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:36.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85500a2130 0x7f85500a45f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f856819a820 tx=0x7f8558005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:36.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 msgr2=0x7f8568199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:36.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f8568199310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f855c000c00 tx=0x7f855c005e80 comp rx=0 tx=0).stop 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 shutdown_connections 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8568104e50 0x7f85681a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85500a2130 0x7f85500a45f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8568104490 0x7f8568199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 --2- 172.21.15.6:0/2210114129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85681032a0 0x7f8568199310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 >> 172.21.15.6:0/2210114129 conn(0x7f85680fe830 msgr2=0x7f8568107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 shutdown_connections 2024-02-26T06:24:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:36.034+0000 7f856ea00640 1 -- 172.21.15.6:0/2210114129 wait complete. 2024-02-26T06:24:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6832/66780761,v1:172.21.15.167:6833/66780761] up:boot 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: fsmap 1 up:standby 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:35 smithi167 ceph-mon[36229]: Deploying daemon mds.k on smithi090 2024-02-26T06:24:37.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:36 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2210114129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:37.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:36 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2210114129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:37.281 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:36 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2210114129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:37.282 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 systemd[1]: Starting Ceph mds.k for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:37.593 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 podman[66374]: 2024-02-26 06:24:37.18488541 +0000 UTC m=+0.017864573 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:37.594 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 podman[66374]: 2024-02-26 06:24:37.289715629 +0000 UTC m=+0.122694798 container create 4d0a8646502364846efe3a4afa9c3058b8105128943f042d52735c4f95321c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-26T06:24:37.594 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 podman[66374]: 2024-02-26 06:24:37.492425827 +0000 UTC m=+0.325404975 container init 4d0a8646502364846efe3a4afa9c3058b8105128943f042d52735c4f95321c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:24:37.594 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 podman[66374]: 2024-02-26 06:24:37.495008416 +0000 UTC m=+0.327987575 container start 4d0a8646502364846efe3a4afa9c3058b8105128943f042d52735c4f95321c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2) 2024-02-26T06:24:37.594 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mds[66393]: -- 172.21.15.90:0/1699974070 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c7a04501a0 con 0x55c7a0427000 2024-02-26T06:24:37.594 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k[66389]: starting mds.k at 2024-02-26T06:24:37.594 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 bash[66374]: 4d0a8646502364846efe3a4afa9c3058b8105128943f042d52735c4f95321c9a 2024-02-26T06:24:37.865 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:24:37 smithi090 systemd[1]: Started Ceph mds.k for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:38.057 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:38.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:37 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:38.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:38.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:38.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:37 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:38.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 -- 172.21.15.6:0/4260500949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564106e90 msgr2=0x7f45641092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:38.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 --2- 172.21.15.6:0/4260500949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564106e90 0x7f45641092e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f455c009ca0 tx=0x7f455c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 -- 172.21.15.6:0/4260500949 shutdown_connections 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 --2- 172.21.15.6:0/4260500949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564106e90 0x7f45641092e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 --2- 172.21.15.6:0/4260500949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4564104560 0x7f4564106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 --2- 172.21.15.6:0/4260500949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564101c30 0x7f4564104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 -- 172.21.15.6:0/4260500949 >> 172.21.15.6:0/4260500949 conn(0x7f45640fb620 msgr2=0x7f45640fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 -- 172.21.15.6:0/4260500949 shutdown_connections 2024-02-26T06:24:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.272+0000 7f4569c00640 1 -- 172.21.15.6:0/4260500949 wait complete. 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.273+0000 7f4569c00640 1 Processor -- start 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.273+0000 7f4569c00640 1 -- start start 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4569c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4564101c30 0x7f45641015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4569c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 0x7f45640ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4569c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564106e90 0x7f4564100120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4569c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456419b010 con 0x7f4564104560 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4569c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564100660 con 0x7f4564106e90 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4569c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564100820 con 0x7f4564101c30 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 0x7f45640ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 0x7f45640ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38960/0 (socket says 172.21.15.6:38960) 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 -- 172.21.15.6:0/4229965735 learned_addr learned my addr 172.21.15.6:0/4229965735 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4563400640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564106e90 0x7f4564100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:38.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 -- 172.21.15.6:0/4229965735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4564101c30 msgr2=0x7f45641015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4564101c30 0x7f45641015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 -- 172.21.15.6:0/4229965735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564106e90 msgr2=0x7f4564100120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564106e90 0x7f4564100120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4562000640 1 -- 172.21.15.6:0/4229965735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c009900 con 0x7f4564104560 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.274+0000 7f4563400640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564106e90 0x7f4564100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.275+0000 7f4562000640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 0x7f45640ffbe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e9f0 tx=0x7f454c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.275+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c0096c0 con 0x7f4564104560 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.275+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f454c005500 con 0x7f4564104560 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.275+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45641a5a00 con 0x7f4564104560 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.275+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c010430 con 0x7f4564104560 2024-02-26T06:24:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.275+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45641a5dc0 con 0x7f4564104560 2024-02-26T06:24:38.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.276+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4564068000 con 0x7f4564104560 2024-02-26T06:24:38.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.278+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f454c005020 con 0x7f4564104560 2024-02-26T06:24:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.279+0000 7f455b400640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f45480a2140 0x7f45480a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.280+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f454c014070 con 0x7f4564104560 2024-02-26T06:24:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.280+0000 7f4562a00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f45480a2140 0x7f45480a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.280+0000 7f4562a00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f45480a2140 0x7f45480a4600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f45540058c0 tx=0x7f4554005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:38.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.283+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f454c097000 con 0x7f4564104560 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:37 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:38.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.575+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45640fd110 con 0x7f4564104560 2024-02-26T06:24:38.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.575+0000 7f455b400640 1 -- 172.21.15.6:0/4229965735 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2110 (secure 0 0 0) 0x7f454c09a6c0 con 0x7f4564104560 2024-02-26T06:24:38.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:38.576 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T06:24:38.576 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 3 2024-02-26T06:24:38.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f45480a2140 msgr2=0x7f45480a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f45480a2140 0x7f45480a4600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f45540058c0 tx=0x7f4554005850 comp rx=0 tx=0).stop 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 msgr2=0x7f45640ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 0x7f45640ffbe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e9f0 tx=0x7f454c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 shutdown_connections 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f45480a2140 0x7f45480a4600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4564106e90 0x7f4564100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4564104560 0x7f45640ffbe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 --2- 172.21.15.6:0/4229965735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4564101c30 0x7f45641015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 >> 172.21.15.6:0/4229965735 conn(0x7f45640fb620 msgr2=0x7f4564102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 shutdown_connections 2024-02-26T06:24:38.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:38.578+0000 7f4569c00640 1 -- 172.21.15.6:0/4229965735 wait complete. 2024-02-26T06:24:39.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:38 smithi006 ceph-mon[31790]: Deploying daemon mds.j on smithi006 2024-02-26T06:24:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:38 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6832/707593181,v1:172.21.15.90:6833/707593181] up:boot 2024-02-26T06:24:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:38 smithi006 ceph-mon[31790]: fsmap 2 up:standby 2024-02-26T06:24:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:38 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:24:39.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:38 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4229965735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:39.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:38 smithi167 ceph-mon[36229]: Deploying daemon mds.j on smithi006 2024-02-26T06:24:39.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:38 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6832/707593181,v1:172.21.15.90:6833/707593181] up:boot 2024-02-26T06:24:39.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:38 smithi167 ceph-mon[36229]: fsmap 2 up:standby 2024-02-26T06:24:39.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:38 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:24:39.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:38 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4229965735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:39.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:38 smithi090 ceph-mon[35681]: Deploying daemon mds.j on smithi006 2024-02-26T06:24:39.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:38 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6832/707593181,v1:172.21.15.90:6833/707593181] up:boot 2024-02-26T06:24:39.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:38 smithi090 ceph-mon[35681]: fsmap 2 up:standby 2024-02-26T06:24:39.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:38 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:24:39.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:38 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4229965735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:39.915 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:39 smithi090 ceph-mon[35681]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:40.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:39 smithi006 ceph-mon[31790]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:40.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:39 smithi167 ceph-mon[36229]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:40.597 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 -- 172.21.15.6:0/166035565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c104570 msgr2=0x7f206c106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 --2- 172.21.15.6:0/166035565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c104570 0x7f206c106960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2054009860 tx=0x7f205402bb70 comp rx=0 tx=0).stop 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 -- 172.21.15.6:0/166035565 shutdown_connections 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 --2- 172.21.15.6:0/166035565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 --2- 172.21.15.6:0/166035565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c104570 0x7f206c106960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 --2- 172.21.15.6:0/166035565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c101c40 0x7f206c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.845+0000 7f2072400640 1 -- 172.21.15.6:0/166035565 >> 172.21.15.6:0/166035565 conn(0x7f206c0fb610 msgr2=0x7f206c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.846+0000 7f2072400640 1 -- 172.21.15.6:0/166035565 shutdown_connections 2024-02-26T06:24:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.846+0000 7f2072400640 1 -- 172.21.15.6:0/166035565 wait complete. 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 Processor -- start 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 -- start start 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 0x7f206c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c104570 0x7f206c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1a0f90 con 0x7f206c106ea0 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1a1100 con 0x7f206c101c40 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f2072400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1a12c0 con 0x7f206c104570 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f206be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.847+0000 7f206b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 0x7f206c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c104570 0x7f206c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 0x7f206c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34338/0 (socket says 172.21.15.6:34338) 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38978/0 (socket says 172.21.15.6:38978) 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 -- 172.21.15.6:0/67873714 learned_addr learned my addr 172.21.15.6:0/67873714 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 -- 172.21.15.6:0/67873714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c104570 msgr2=0x7f206c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c104570 0x7f206c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 -- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 msgr2=0x7f206c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 -- 172.21.15.6:0/67873714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009490 con 0x7f206c101c40 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206aa00640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c104570 0x7f206c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206be00640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f206b400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 0x7f206c1994a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f205c00eb90 tx=0x7f205c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.848+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00cea0 con 0x7f206c101c40 2024-02-26T06:24:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.849+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f205c005020 con 0x7f206c101c40 2024-02-26T06:24:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.849+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c010670 con 0x7f206c101c40 2024-02-26T06:24:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.849+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c1a15a0 con 0x7f206c101c40 2024-02-26T06:24:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.849+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1a1a30 con 0x7f206c101c40 2024-02-26T06:24:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.850+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c1089f0 con 0x7f206c101c40 2024-02-26T06:24:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.853+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f205c005190 con 0x7f206c101c40 2024-02-26T06:24:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.854+0000 7f2063e00640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20500a2140 0x7f20500a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.855+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f205c014070 con 0x7f206c101c40 2024-02-26T06:24:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.855+0000 7f206aa00640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20500a2140 0x7f20500a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.855+0000 7f206aa00640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20500a2140 0x7f20500a4600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2054009830 tx=0x7f2054034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:40.858+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f205c09ad60 con 0x7f206c101c40 2024-02-26T06:24:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.111+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f206c10cbd0 con 0x7f206c101c40 2024-02-26T06:24:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.112+0000 7f2063e00640 1 -- 172.21.15.6:0/67873714 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2110 (secure 0 0 0) 0x7f205c09a8f0 con 0x7f206c101c40 2024-02-26T06:24:41.113 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:41.113 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T06:24:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 3 2024-02-26T06:24:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20500a2140 msgr2=0x7f20500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20500a2140 0x7f20500a4600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2054009830 tx=0x7f2054034c00 comp rx=0 tx=0).stop 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 msgr2=0x7f206c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 0x7f206c1994a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f205c00eb90 tx=0x7f205c00ef90 comp rx=0 tx=0).stop 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 shutdown_connections 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f206c106ea0 0x7f206c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f206c104570 0x7f206c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20500a2140 0x7f20500a4600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 --2- 172.21.15.6:0/67873714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f206c101c40 0x7f206c1994a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 >> 172.21.15.6:0/67873714 conn(0x7f206c0fb610 msgr2=0x7f206c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 shutdown_connections 2024-02-26T06:24:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:41.115+0000 7f2072400640 1 -- 172.21.15.6:0/67873714 wait complete. 2024-02-26T06:24:42.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:41 smithi006 ceph-mon[31790]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:42.467 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:41 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/67873714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:42.468 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:41 smithi090 ceph-mon[35681]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:42.468 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:41 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/67873714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:42.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:41 smithi167 ceph-mon[36229]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:42.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:41 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/67873714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6834/1064493742,v1:172.21.15.6:6835/1064493742] up:boot 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: fsmap 3 up:standby 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.068 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:43.069 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:43.069 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:43.069 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:42 smithi006 ceph-mon[31790]: Deploying daemon mds.d on smithi006 2024-02-26T06:24:43.136 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:43.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6834/1064493742,v1:172.21.15.6:6835/1064493742] up:boot 2024-02-26T06:24:43.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: fsmap 3 up:standby 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:43.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:42 smithi167 ceph-mon[36229]: Deploying daemon mds.d on smithi006 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6834/1064493742,v1:172.21.15.6:6835/1064493742] up:boot 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: fsmap 3 up:standby 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:43.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:43.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:43.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:42 smithi090 ceph-mon[35681]: Deploying daemon mds.d on smithi006 2024-02-26T06:24:43.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 -- 172.21.15.6:0/2840063185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 msgr2=0x7f79b4102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:43.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 --2- 172.21.15.6:0/2840063185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4102880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ca0 tx=0x7f799c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 -- 172.21.15.6:0/2840063185 shutdown_connections 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 --2- 172.21.15.6:0/2840063185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4102dc0 0x7f79b4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 --2- 172.21.15.6:0/2840063185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4102880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 --2- 172.21.15.6:0/2840063185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b4101210 0x7f79b4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 -- 172.21.15.6:0/2840063185 >> 172.21.15.6:0/2840063185 conn(0x7f79b40fc7a0 msgr2=0x7f79b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 -- 172.21.15.6:0/2840063185 shutdown_connections 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.373+0000 7f79bb400640 1 -- 172.21.15.6:0/2840063185 wait complete. 2024-02-26T06:24:43.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.374+0000 7f79bb400640 1 Processor -- start 2024-02-26T06:24:43.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 -- start start 2024-02-26T06:24:43.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4101210 0x7f79b4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b4102dc0 0x7f79b41a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41a0f20 con 0x7f79b4102400 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41a1090 con 0x7f79b4101210 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79bb400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41a1250 con 0x7f79b4102dc0 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79b3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79b3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39022/0 (socket says 172.21.15.6:39022) 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79b3e00640 1 -- 172.21.15.6:0/3328932009 learned_addr learned my addr 172.21.15.6:0/3328932009 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79b8a00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4101210 0x7f79b4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.375+0000 7f79b9400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b4102dc0 0x7f79b41a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b3e00640 1 -- 172.21.15.6:0/3328932009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b4102dc0 msgr2=0x7f79b41a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b3e00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b4102dc0 0x7f79b41a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b3e00640 1 -- 172.21.15.6:0/3328932009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4101210 msgr2=0x7f79b4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b3e00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4101210 0x7f79b4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b3e00640 1 -- 172.21.15.6:0/3328932009 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c009900 con 0x7f79b4102400 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b8a00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4101210 0x7f79b4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b3e00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4199970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f799c032710 tx=0x7f799c034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0329b0 con 0x7f79b4102400 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799c003e50 con 0x7f79b4102400 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.376+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b41a14d0 con 0x7f79b4102400 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.377+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c040450 con 0x7f79b4102400 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.377+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b41a19a0 con 0x7f79b4102400 2024-02-26T06:24:43.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.378+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b410cbb0 con 0x7f79b4102400 2024-02-26T06:24:43.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.381+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f799c00a040 con 0x7f79b4102400 2024-02-26T06:24:43.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.383+0000 7f79b1600640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79940a2140 0x7f79940a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:43.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.383+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f799c0f0a10 con 0x7f79b4102400 2024-02-26T06:24:43.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.383+0000 7f79b8a00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79940a2140 0x7f79940a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:43.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.384+0000 7f79b8a00640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79940a2140 0x7f79940a4600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79b4102260 tx=0x7f79a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:43.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.385+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f799c0b9e60 con 0x7f79b4102400 2024-02-26T06:24:43.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.619+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79b410cd40 con 0x7f79b4102400 2024-02-26T06:24:43.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.620+0000 7f79b1600640 1 -- 172.21.15.6:0/3328932009 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2898 (secure 0 0 0) 0x7f799c044b70 con 0x7f79b4102400 2024-02-26T06:24:43.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:43.620 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T06:24:43.620 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 4 2024-02-26T06:24:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79940a2140 msgr2=0x7f79940a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79940a2140 0x7f79940a4600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79b4102260 tx=0x7f79a4005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 msgr2=0x7f79b4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4199970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f799c032710 tx=0x7f799c034a90 comp rx=0 tx=0).stop 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 shutdown_connections 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b4102dc0 0x7f79b41a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79b4102400 0x7f79b4199970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f79940a2140 0x7f79940a4600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 --2- 172.21.15.6:0/3328932009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b4101210 0x7f79b4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 >> 172.21.15.6:0/3328932009 conn(0x7f79b40fc7a0 msgr2=0x7f79b40733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 shutdown_connections 2024-02-26T06:24:43.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:43.622+0000 7f79bb400640 1 -- 172.21.15.6:0/3328932009 wait complete. 2024-02-26T06:24:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:43 smithi006 ceph-mon[31790]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:43 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3328932009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:44.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:43 smithi167 ceph-mon[36229]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:43 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3328932009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:43 smithi090 ceph-mon[35681]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:43 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3328932009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:45.422 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 systemd[1]: Starting Ceph mds.d for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:45.640 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:45.819 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 podman[72498]: 2024-02-26 06:24:45.459311084 +0000 UTC m=+0.033323252 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:45.819 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 podman[72498]: 2024-02-26 06:24:45.576346046 +0000 UTC m=+0.150358264 container create 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-26T06:24:45.820 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 podman[72498]: 2024-02-26 06:24:45.805413602 +0000 UTC m=+0.379425837 container init 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-26T06:24:45.820 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 podman[72498]: 2024-02-26 06:24:45.819042144 +0000 UTC m=+0.393054372 container start 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:24:45.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 -- 172.21.15.6:0/108652103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9801044c0 msgr2=0x7fb9801068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:45.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 --2- 172.21.15.6:0/108652103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9801044c0 0x7fb9801068b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb970009860 tx=0x7fb97002bb70 comp rx=0 tx=0).stop 2024-02-26T06:24:45.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 -- 172.21.15.6:0/108652103 shutdown_connections 2024-02-26T06:24:45.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 --2- 172.21.15.6:0/108652103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb980106df0 0x7fb9801091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 --2- 172.21.15.6:0/108652103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9801044c0 0x7fb9801068b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 --2- 172.21.15.6:0/108652103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb980101b90 0x7fb980103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 -- 172.21.15.6:0/108652103 >> 172.21.15.6:0/108652103 conn(0x7fb9800fb500 msgr2=0x7fb9800fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:45.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 -- 172.21.15.6:0/108652103 shutdown_connections 2024-02-26T06:24:45.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.865+0000 7fb987200640 1 -- 172.21.15.6:0/108652103 wait complete. 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.866+0000 7fb987200640 1 Processor -- start 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 -- start start 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 0x7fb980101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9801044c0 0x7fb9800ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb980106df0 0x7fb980100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801007e0 con 0x7fb9801044c0 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb980100950 con 0x7fb980101b90 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb987200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801a1190 con 0x7fb980106df0 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb985400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9801044c0 0x7fb9800ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb985e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 0x7fb980101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb985e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 0x7fb980101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34386/0 (socket says 172.21.15.6:34386) 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb985e00640 1 -- 172.21.15.6:0/2060354509 learned_addr learned my addr 172.21.15.6:0/2060354509 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.867+0000 7fb986800640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb980106df0 0x7fb980100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb985e00640 1 -- 172.21.15.6:0/2060354509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb980106df0 msgr2=0x7fb980100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb985e00640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb980106df0 0x7fb980100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb985e00640 1 -- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9801044c0 msgr2=0x7fb9800ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb985e00640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9801044c0 0x7fb9800ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb985e00640 1 -- 172.21.15.6:0/2060354509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970009490 con 0x7fb980101b90 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb985e00640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 0x7fb980101500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e9f0 tx=0x7fb97400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400ccc0 con 0x7fb980101b90 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb97400ce20 con 0x7fb980101b90 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.868+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974010430 con 0x7fb980101b90 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.869+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9801a1440 con 0x7fb980101b90 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.869+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9801a18d0 con 0x7fb980101b90 2024-02-26T06:24:45.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.870+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9800fcff0 con 0x7fb980101b90 2024-02-26T06:24:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.872+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb9740046c0 con 0x7fb980101b90 2024-02-26T06:24:45.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.873+0000 7fb97aa00640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9680a2130 0x7fb9680a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:45.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.873+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb974014070 con 0x7fb980101b90 2024-02-26T06:24:45.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.874+0000 7fb985400640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9680a2130 0x7fb9680a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:45.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.874+0000 7fb985400640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9680a2130 0x7fb9680a45f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb970034b10 tx=0x7fb970034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:45.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:45.877+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb974096e60 con 0x7fb980101b90 2024-02-26T06:24:46.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.142+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb98010ca70 con 0x7fb980101b90 2024-02-26T06:24:46.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.143+0000 7fb97aa00640 1 -- 172.21.15.6:0/2060354509 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3683 (secure 0 0 0) 0x7fb97409a520 con 0x7fb980101b90 2024-02-26T06:24:46.144 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:46.144 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-02-26T06:24:46.144 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 5 2024-02-26T06:24:46.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9680a2130 msgr2=0x7fb9680a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9680a2130 0x7fb9680a45f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb970034b10 tx=0x7fb970034be0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 msgr2=0x7fb980101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 0x7fb980101500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e9f0 tx=0x7fb97400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 shutdown_connections 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb980106df0 0x7fb980100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9801044c0 0x7fb9800ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9680a2130 0x7fb9680a45f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 --2- 172.21.15.6:0/2060354509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb980101b90 0x7fb980101500 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 >> 172.21.15.6:0/2060354509 conn(0x7fb9800fb500 msgr2=0x7fb9801024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 shutdown_connections 2024-02-26T06:24:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:46.146+0000 7fb987200640 1 -- 172.21.15.6:0/2060354509 wait complete. 2024-02-26T06:24:46.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:45 smithi006 ceph-mon[31790]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:46.165 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 ceph-mds[72536]: -- 172.21.15.6:0/2006736950 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b7194c61a0 con 0x55b71949c800 2024-02-26T06:24:46.165 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d[72525]: starting mds.d at 2024-02-26T06:24:46.165 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 bash[72498]: 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 2024-02-26T06:24:46.165 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:24:45 smithi006 systemd[1]: Started Ceph mds.d for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:46.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:45 smithi167 ceph-mon[36229]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:46.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:45 smithi090 ceph-mon[35681]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:47.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6836/234881158,v1:172.21.15.6:6837/234881158] up:boot 2024-02-26T06:24:47.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: fsmap 4 up:standby 2024-02-26T06:24:47.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:24:47.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.352 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.352 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:47.352 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:47.352 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:47.352 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:46 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2060354509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6836/234881158,v1:172.21.15.6:6837/234881158] up:boot 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: fsmap 4 up:standby 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:47.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:47.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:47.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:46 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2060354509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:47.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6836/234881158,v1:172.21.15.6:6837/234881158] up:boot 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: fsmap 4 up:standby 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:47.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:46 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2060354509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:48.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:48.256 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:47 smithi006 ceph-mon[31790]: Deploying daemon mds.e on smithi090 2024-02-26T06:24:48.256 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:47 smithi006 ceph-mon[31790]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:48.290 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:47 smithi090 ceph-mon[35681]: Deploying daemon mds.e on smithi090 2024-02-26T06:24:48.290 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:47 smithi090 ceph-mon[35681]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:48.290 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 systemd[1]: Starting Ceph mds.e for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:48.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 -- 172.21.15.6:0/3797473189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c073b80 msgr2=0x7f404c073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:48.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 --2- 172.21.15.6:0/3797473189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c073b80 0x7f404c073fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4038009900 tx=0x7f403802bcd0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 -- 172.21.15.6:0/3797473189 shutdown_connections 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 --2- 172.21.15.6:0/3797473189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f404c1090a0 0x7f404c10b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 --2- 172.21.15.6:0/3797473189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c073b80 0x7f404c073fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 --2- 172.21.15.6:0/3797473189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f404c075160 0x7f404c0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 -- 172.21.15.6:0/3797473189 >> 172.21.15.6:0/3797473189 conn(0x7f404c0fc760 msgr2=0x7f404c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 -- 172.21.15.6:0/3797473189 shutdown_connections 2024-02-26T06:24:48.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.400+0000 7f4050e00640 1 -- 172.21.15.6:0/3797473189 wait complete. 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 Processor -- start 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 -- start start 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f404c073b80 0x7f404c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f404c075160 0x7f404c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 0x7f404c1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1a0f20 con 0x7f404c1090a0 2024-02-26T06:24:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.401+0000 7f4050e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1a1090 con 0x7f404c073b80 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f4050e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1a1250 con 0x7f404c075160 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 0x7f404c1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 0x7f404c1a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48112/0 (socket says 172.21.15.6:48112) 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 -- 172.21.15.6:0/3558602702 learned_addr learned my addr 172.21.15.6:0/3558602702 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f4049400640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f404c075160 0x7f404c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 -- 172.21.15.6:0/3558602702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f404c075160 msgr2=0x7f404c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f404c075160 0x7f404c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 -- 172.21.15.6:0/3558602702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f404c073b80 msgr2=0x7f404c199430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f404c073b80 0x7f404c199430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.402+0000 7f404a800640 1 -- 172.21.15.6:0/3558602702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4044009840 con 0x7f404c1090a0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.403+0000 7f404a800640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 0x7f404c1a09e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f404400efd0 tx=0x7f404400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.403+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044009b40 con 0x7f404c1090a0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.403+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4038009560 con 0x7f404c1090a0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.403+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4044003b70 con 0x7f404c1090a0 2024-02-26T06:24:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.403+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40440108f0 con 0x7f404c1090a0 2024-02-26T06:24:48.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.403+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c1a1830 con 0x7f404c1090a0 2024-02-26T06:24:48.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.404+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c107e60 con 0x7f404c1090a0 2024-02-26T06:24:48.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.406+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4044003ce0 con 0x7f404c1090a0 2024-02-26T06:24:48.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.407+0000 7f4042a00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f40300a20c0 0x7f40300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:48.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.408+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4044014070 con 0x7f404c1090a0 2024-02-26T06:24:48.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.408+0000 7f4049e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f40300a20c0 0x7f40300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:48.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.408+0000 7f4049e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f40300a20c0 0x7f40300a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f403c009a60 tx=0x7f403c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:48.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.413+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40440972e0 con 0x7f404c1090a0 2024-02-26T06:24:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:47 smithi167 ceph-mon[36229]: Deploying daemon mds.e on smithi090 2024-02-26T06:24:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:47 smithi167 ceph-mon[36229]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:48.592 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 podman[66695]: 2024-02-26 06:24:48.193317004 +0000 UTC m=+0.017119920 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:48.592 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 podman[66695]: 2024-02-26 06:24:48.306829439 +0000 UTC m=+0.130632371 container create 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-26T06:24:48.593 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 podman[66695]: 2024-02-26 06:24:48.492631453 +0000 UTC m=+0.316434366 container init 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-26T06:24:48.593 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 podman[66695]: 2024-02-26 06:24:48.495235079 +0000 UTC m=+0.319037992 container start 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:24:48.593 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 ceph-mds[66715]: -- 172.21.15.90:0/3920719375 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5618a76c81a0 con 0x5618a769f000 2024-02-26T06:24:48.593 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e[66711]: starting mds.e at 2024-02-26T06:24:48.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.656+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f404c02ce40 con 0x7f404c1090a0 2024-02-26T06:24:48.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.660+0000 7f4042a00640 1 -- 172.21.15.6:0/3558602702 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3683 (secure 0 0 0) 0x7f404409a9a0 con 0x7f404c1090a0 2024-02-26T06:24:48.660 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:48.660 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-02-26T06:24:48.661 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 5 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f40300a20c0 msgr2=0x7f40300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f40300a20c0 0x7f40300a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f403c009a60 tx=0x7f403c005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 msgr2=0x7f404c1a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 0x7f404c1a09e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f404400efd0 tx=0x7f404400c6a0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 shutdown_connections 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c1090a0 0x7f404c1a09e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f404c075160 0x7f404c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f40300a20c0 0x7f40300a4580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 --2- 172.21.15.6:0/3558602702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f404c073b80 0x7f404c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:48.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 >> 172.21.15.6:0/3558602702 conn(0x7f404c0fc760 msgr2=0x7f404c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:48.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 shutdown_connections 2024-02-26T06:24:48.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:48.663+0000 7f4050e00640 1 -- 172.21.15.6:0/3558602702 wait complete. 2024-02-26T06:24:48.865 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 bash[66695]: 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 2024-02-26T06:24:48.865 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:24:48 smithi090 systemd[1]: Started Ceph mds.e for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:49.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:49.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:49.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3558602702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: Deploying daemon mds.f on smithi167 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6834/679395830,v1:172.21.15.90:6835/679395830] up:boot 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: fsmap 5 up:standby 2024-02-26T06:24:49.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:24:49.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3558602702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: Deploying daemon mds.f on smithi167 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6834/679395830,v1:172.21.15.90:6835/679395830] up:boot 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: fsmap 5 up:standby 2024-02-26T06:24:49.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3558602702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:50.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:50.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: Deploying daemon mds.f on smithi167 2024-02-26T06:24:50.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6834/679395830,v1:172.21.15.90:6835/679395830] up:boot 2024-02-26T06:24:50.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: fsmap 5 up:standby 2024-02-26T06:24:50.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:24:50.685 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 -- 172.21.15.6:0/4130058129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48101080 msgr2=0x7f0a481014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/4130058129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48101080 0x7f0a481014a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009900 tx=0x7f0a3802bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 -- 172.21.15.6:0/4130058129 shutdown_connections 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/4130058129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48102c30 0x7f0a481094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/4130058129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48102270 0x7f0a481026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/4130058129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48101080 0x7f0a481014a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 -- 172.21.15.6:0/4130058129 >> 172.21.15.6:0/4130058129 conn(0x7f0a480fc610 msgr2=0x7f0a480fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.916+0000 7f0a4ce00640 1 -- 172.21.15.6:0/4130058129 shutdown_connections 2024-02-26T06:24:50.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.917+0000 7f0a4ce00640 1 -- 172.21.15.6:0/4130058129 wait complete. 2024-02-26T06:24:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.918+0000 7f0a4ce00640 1 Processor -- start 2024-02-26T06:24:50.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.918+0000 7f0a4ce00640 1 -- start start 2024-02-26T06:24:50.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a4ce00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48101080 0x7f0a48199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:50.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a4ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 0x7f0a481998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a4ce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48102c30 0x7f0a481a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a4ce00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481a0e50 con 0x7f0a48101080 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a4ce00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481a0fc0 con 0x7f0a48102c30 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a4ce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481a1180 con 0x7f0a48102270 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a47400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48101080 0x7f0a48199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 0x7f0a481998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 0x7f0a481998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34480/0 (socket says 172.21.15.6:34480) 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a47400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48101080 0x7f0a48199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48134/0 (socket says 172.21.15.6:48134) 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 -- 172.21.15.6:0/434690488 learned_addr learned my addr 172.21.15.6:0/434690488 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a47e00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48102c30 0x7f0a481a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 -- 172.21.15.6:0/434690488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48102c30 msgr2=0x7f0a481a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48102c30 0x7f0a481a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 -- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48101080 msgr2=0x7f0a48199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:50.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48101080 0x7f0a48199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.919+0000 7f0a46a00640 1 -- 172.21.15.6:0/434690488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38009560 con 0x7f0a48102270 2024-02-26T06:24:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.920+0000 7f0a46a00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 0x7f0a481998a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000d950 tx=0x7f0a3000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:50.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.920+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30009a30 con 0x7f0a48102270 2024-02-26T06:24:50.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.920+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a30009b90 con 0x7f0a48102270 2024-02-26T06:24:50.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.920+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a481a1460 con 0x7f0a48102270 2024-02-26T06:24:50.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.920+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004a00 con 0x7f0a48102270 2024-02-26T06:24:50.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.920+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a481a1970 con 0x7f0a48102270 2024-02-26T06:24:50.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.921+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4810cae0 con 0x7f0a48102270 2024-02-26T06:24:50.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.923+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0a30009d00 con 0x7f0a48102270 2024-02-26T06:24:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.925+0000 7f0a3fe00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a2c0a20b0 0x7f0a2c0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.925+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a300cd270 con 0x7f0a48102270 2024-02-26T06:24:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.925+0000 7f0a47400640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a2c0a20b0 0x7f0a2c0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.926+0000 7f0a47400640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a2c0a20b0 0x7f0a2c0a4570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a3802c240 tx=0x7f0a3802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:50.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:50.927+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a30096640 con 0x7f0a48102270 2024-02-26T06:24:51.157 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:50 smithi167 systemd[1]: Starting Ceph mds.f for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:51.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.168+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a4810cc70 con 0x7f0a48102270 2024-02-26T06:24:51.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.169+0000 7f0a3fe00640 1 -- 172.21.15.6:0/434690488 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4471 (secure 0 0 0) 0x7f0a30099d00 con 0x7f0a48102270 2024-02-26T06:24:51.169 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:51.169 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-26T06:24:51.170 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 6 2024-02-26T06:24:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a2c0a20b0 msgr2=0x7f0a2c0a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a2c0a20b0 0x7f0a2c0a4570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a3802c240 tx=0x7f0a3802c7e0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 msgr2=0x7f0a481998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 0x7f0a481998a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000d950 tx=0x7f0a3000de10 comp rx=0 tx=0).stop 2024-02-26T06:24:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 shutdown_connections 2024-02-26T06:24:51.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0a2c0a20b0 0x7f0a2c0a4570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48102c30 0x7f0a481a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a48102270 0x7f0a481998a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 --2- 172.21.15.6:0/434690488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a48101080 0x7f0a48199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 >> 172.21.15.6:0/434690488 conn(0x7f0a480fc610 msgr2=0x7f0a480733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 shutdown_connections 2024-02-26T06:24:51.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:51.172+0000 7f0a4ce00640 1 -- 172.21.15.6:0/434690488 wait complete. 2024-02-26T06:24:51.424 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 podman[67130]: 2024-02-26 06:24:51.06040395 +0000 UTC m=+0.018095993 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:51.425 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 podman[67130]: 2024-02-26 06:24:51.164383226 +0000 UTC m=+0.122075295 container create bee6b06b91a6a72f7609c818fa40397c8131776b0a4b737d42aec1ee4d23780a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T06:24:51.425 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 podman[67130]: 2024-02-26 06:24:51.425658282 +0000 UTC m=+0.383350311 container init bee6b06b91a6a72f7609c818fa40397c8131776b0a4b737d42aec1ee4d23780a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-26T06:24:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/434690488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 podman[67130]: 2024-02-26 06:24:51.428321976 +0000 UTC m=+0.386014006 container start bee6b06b91a6a72f7609c818fa40397c8131776b0a4b737d42aec1ee4d23780a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-mds[67149]: -- 172.21.15.167:0/666435362 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559daa9a81a0 con 0x559daa97ec00 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f[67145]: starting mds.f at 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 bash[67130]: bee6b06b91a6a72f7609c818fa40397c8131776b0a4b737d42aec1ee4d23780a 2024-02-26T06:24:51.745 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:24:51 smithi167 systemd[1]: Started Ceph mds.f for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/434690488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/434690488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:52.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:52 smithi167 ceph-mon[36229]: Deploying daemon mds.c on smithi167 2024-02-26T06:24:52.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:52 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6834/2489701568,v1:172.21.15.167:6835/2489701568] up:boot 2024-02-26T06:24:52.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:52 smithi167 ceph-mon[36229]: fsmap 6 up:standby 2024-02-26T06:24:52.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:24:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:52 smithi090 ceph-mon[35681]: Deploying daemon mds.c on smithi167 2024-02-26T06:24:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:52 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6834/2489701568,v1:172.21.15.167:6835/2489701568] up:boot 2024-02-26T06:24:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:52 smithi090 ceph-mon[35681]: fsmap 6 up:standby 2024-02-26T06:24:53.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:24:53.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:52 smithi006 ceph-mon[31790]: Deploying daemon mds.c on smithi167 2024-02-26T06:24:53.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:52 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6834/2489701568,v1:172.21.15.167:6835/2489701568] up:boot 2024-02-26T06:24:53.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:52 smithi006 ceph-mon[31790]: fsmap 6 up:standby 2024-02-26T06:24:53.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:24:53.194 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:53.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.416+0000 7f336d000640 1 -- 172.21.15.6:0/4184137707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 msgr2=0x7f336810b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.416+0000 7f336d000640 1 --2- 172.21.15.6:0/4184137707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f336810b6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3360009ca0 tx=0x7f336002c100 comp rx=0 tx=0).stop 2024-02-26T06:24:53.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 -- 172.21.15.6:0/4184137707 shutdown_connections 2024-02-26T06:24:53.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 --2- 172.21.15.6:0/4184137707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f336810b6e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 --2- 172.21.15.6:0/4184137707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3368104490 0x7f3368104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 --2- 172.21.15.6:0/4184137707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33681032a0 0x7f33681036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 -- 172.21.15.6:0/4184137707 >> 172.21.15.6:0/4184137707 conn(0x7f33680fe830 msgr2=0x7f3368100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 -- 172.21.15.6:0/4184137707 shutdown_connections 2024-02-26T06:24:53.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.417+0000 7f336d000640 1 -- 172.21.15.6:0/4184137707 wait complete. 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.418+0000 7f336d000640 1 Processor -- start 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.418+0000 7f336d000640 1 -- start start 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f336d000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33681032a0 0x7f3368199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f336d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3368104490 0x7f33681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f336d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f33681a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f336d000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33681a0d50 con 0x7f33681032a0 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f336d000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33681a0ec0 con 0x7f3368104490 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f336d000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33681a1080 con 0x7f3368104e50 2024-02-26T06:24:53.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33681032a0 0x7f3368199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f33681a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f33681a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34502/0 (socket says 172.21.15.6:34502) 2024-02-26T06:24:53.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 -- 172.21.15.6:0/4068330450 learned_addr learned my addr 172.21.15.6:0/4068330450 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:53.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f335fe00640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3368104490 0x7f33681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 -- 172.21.15.6:0/4068330450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3368104490 msgr2=0x7f33681997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3368104490 0x7f33681997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 -- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33681032a0 msgr2=0x7f3368199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33681032a0 0x7f3368199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.419+0000 7f3367e00640 1 -- 172.21.15.6:0/4068330450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360009900 con 0x7f3368104e50 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.420+0000 7f3367e00640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f33681a0810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3360037040 tx=0x7f3360034bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.420+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360034850 con 0x7f3368104e50 2024-02-26T06:24:53.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.420+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3360034de0 con 0x7f3368104e50 2024-02-26T06:24:53.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.420+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360040450 con 0x7f3368104e50 2024-02-26T06:24:53.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.420+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33681a1300 con 0x7f3368104e50 2024-02-26T06:24:53.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.420+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33681a1760 con 0x7f3368104e50 2024-02-26T06:24:53.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.421+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336810a560 con 0x7f3368104e50 2024-02-26T06:24:53.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.423+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f336000a040 con 0x7f3368104e50 2024-02-26T06:24:53.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.425+0000 7f3364c00640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f334c0a2130 0x7f334c0a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:53.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.425+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f33600f0310 con 0x7f3368104e50 2024-02-26T06:24:53.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.425+0000 7f3367400640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f334c0a2130 0x7f334c0a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:53.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.425+0000 7f3367400640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f334c0a2130 0x7f334c0a45f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33681042f0 tx=0x7f3358005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:53.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.429+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33600b96e0 con 0x7f3368104e50 2024-02-26T06:24:53.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.673+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f336802cda0 con 0x7f3368104e50 2024-02-26T06:24:53.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.681+0000 7f3364c00640 1 -- 172.21.15.6:0/4068330450 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5265 (secure 0 0 0) 0x7f33600bcda0 con 0x7f3368104e50 2024-02-26T06:24:53.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:53.681 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-26T06:24:53.681 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 7 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f334c0a2130 msgr2=0x7f334c0a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f334c0a2130 0x7f334c0a45f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33681042f0 tx=0x7f3358005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 msgr2=0x7f33681a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f33681a0810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3360037040 tx=0x7f3360034bf0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 shutdown_connections 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3368104e50 0x7f33681a0810 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f334c0a2130 0x7f334c0a45f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3368104490 0x7f33681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 --2- 172.21.15.6:0/4068330450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33681032a0 0x7f3368199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 >> 172.21.15.6:0/4068330450 conn(0x7f33680fe830 msgr2=0x7f3368107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 shutdown_connections 2024-02-26T06:24:53.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:53.684+0000 7f336d000640 1 -- 172.21.15.6:0/4068330450 wait complete. 2024-02-26T06:24:53.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:53 smithi167 ceph-mon[36229]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:53.995 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:53 smithi167 systemd[1]: Starting Ceph mds.c for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:54.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:53 smithi090 ceph-mon[35681]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:54.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:53 smithi006 ceph-mon[31790]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:54.271 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:53 smithi167 podman[67436]: 2024-02-26 06:24:53.901136714 +0000 UTC m=+0.017882169 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:54.271 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 podman[67436]: 2024-02-26 06:24:54.036025213 +0000 UTC m=+0.152770666 container create a5841f2d50c2a3bbe8bb48bc99c399ff597e9554573a26dca99c9e7ff0c01348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:24:54.271 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 podman[67436]: 2024-02-26 06:24:54.271918323 +0000 UTC m=+0.388663774 container init a5841f2d50c2a3bbe8bb48bc99c399ff597e9554573a26dca99c9e7ff0c01348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:24:54.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 podman[67436]: 2024-02-26 06:24:54.274739797 +0000 UTC m=+0.391485253 container start a5841f2d50c2a3bbe8bb48bc99c399ff597e9554573a26dca99c9e7ff0c01348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:24:54.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mds[67455]: -- 172.21.15.167:0/605668854 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a2b6781a0 con 0x564a2b64f000 2024-02-26T06:24:54.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c[67451]: starting mds.c at 2024-02-26T06:24:54.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 bash[67436]: a5841f2d50c2a3bbe8bb48bc99c399ff597e9554573a26dca99c9e7ff0c01348 2024-02-26T06:24:54.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:24:54 smithi167 systemd[1]: Started Ceph mds.c for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:54.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4068330450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:54.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:54.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:54.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:54.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:54.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:54.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:54 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4068330450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:55.105 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:54 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4068330450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:55.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:54 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:55.704 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.934+0000 7ff009600640 1 -- 172.21.15.6:0/477022459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 msgr2=0x7ff004107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.934+0000 7ff009600640 1 --2- 172.21.15.6:0/477022459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff004107be0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feff4009900 tx=0x7feff402bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.934+0000 7ff009600640 1 -- 172.21.15.6:0/477022459 shutdown_connections 2024-02-26T06:24:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.934+0000 7ff009600640 1 --2- 172.21.15.6:0/477022459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff004109370 0x7ff00410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.935+0000 7ff009600640 1 --2- 172.21.15.6:0/477022459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0041089b0 0x7ff004108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.935+0000 7ff009600640 1 --2- 172.21.15.6:0/477022459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff004107be0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.935+0000 7ff009600640 1 -- 172.21.15.6:0/477022459 >> 172.21.15.6:0/477022459 conn(0x7ff004076780 msgr2=0x7ff004078ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.935+0000 7ff009600640 1 -- 172.21.15.6:0/477022459 shutdown_connections 2024-02-26T06:24:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.935+0000 7ff009600640 1 -- 172.21.15.6:0/477022459 wait complete. 2024-02-26T06:24:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 Processor -- start 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 -- start start 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff00419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0041089b0 0x7ff00419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff004109370 0x7ff0041a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a5350 con 0x7ff0041077c0 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a54c0 con 0x7ff004109370 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff009600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a5680 con 0x7ff0041089b0 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff002a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff00419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff002a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff00419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48172/0 (socket says 172.21.15.6:48172) 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff002a00640 1 -- 172.21.15.6:0/3254089467 learned_addr learned my addr 172.21.15.6:0/3254089467 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff002000640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0041089b0 0x7ff00419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.936+0000 7ff003400640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff004109370 0x7ff0041a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff002a00640 1 -- 172.21.15.6:0/3254089467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0041089b0 msgr2=0x7ff00419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff002a00640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0041089b0 0x7ff00419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff002a00640 1 -- 172.21.15.6:0/3254089467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff004109370 msgr2=0x7ff0041a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff002a00640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff004109370 0x7ff0041a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff002a00640 1 -- 172.21.15.6:0/3254089467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4009560 con 0x7ff0041077c0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff002a00640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff00419d860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40043d0 con 0x7ff0041077c0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feff40052d0 con 0x7ff0041077c0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4003da0 con 0x7ff0041077c0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.937+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0041a5900 con 0x7ff0041077c0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.938+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a5e60 con 0x7ff0041077c0 2024-02-26T06:24:55.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.939+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00410eb60 con 0x7ff0041077c0 2024-02-26T06:24:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.941+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7feff4005440 con 0x7ff0041077c0 2024-02-26T06:24:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.942+0000 7feffb400640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fefe80a2140 0x7fefe80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.942+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feff40f0370 con 0x7ff0041077c0 2024-02-26T06:24:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.943+0000 7ff002000640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fefe80a2140 0x7fefe80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.943+0000 7ff002000640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fefe80a2140 0x7fefe80a4600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff00419ed70 tx=0x7fefec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:55.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:55.945+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feff40b9740 con 0x7ff0041077c0 2024-02-26T06:24:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:55 smithi167 ceph-mon[36229]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:55 smithi167 ceph-mon[36229]: Deploying daemon mds.h on smithi090 2024-02-26T06:24:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:55 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] up:boot 2024-02-26T06:24:55.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:55 smithi167 ceph-mon[36229]: fsmap 7 up:standby 2024-02-26T06:24:55.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:55 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:24:56.068 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:55 smithi090 ceph-mon[35681]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:56.072 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:55 smithi090 ceph-mon[35681]: Deploying daemon mds.h on smithi090 2024-02-26T06:24:56.072 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:55 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] up:boot 2024-02-26T06:24:56.072 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:55 smithi090 ceph-mon[35681]: fsmap 7 up:standby 2024-02-26T06:24:56.072 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:55 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:24:56.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:55 smithi006 ceph-mon[31790]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:56.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:55 smithi006 ceph-mon[31790]: Deploying daemon mds.h on smithi090 2024-02-26T06:24:56.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:55 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] up:boot 2024-02-26T06:24:56.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:55 smithi006 ceph-mon[31790]: fsmap 7 up:standby 2024-02-26T06:24:56.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:55 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:24:56.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.187+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff004113210 con 0x7ff0041077c0 2024-02-26T06:24:56.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.189+0000 7feffb400640 1 -- 172.21.15.6:0/3254089467 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6053 (secure 0 0 0) 0x7feff40bce00 con 0x7ff0041077c0 2024-02-26T06:24:56.189 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:56.190 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-26T06:24:56.190 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 8 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fefe80a2140 msgr2=0x7fefe80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fefe80a2140 0x7fefe80a4600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff00419ed70 tx=0x7fefec005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 msgr2=0x7ff00419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff00419d860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff4005e80 comp rx=0 tx=0).stop 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 shutdown_connections 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fefe80a2140 0x7fefe80a4600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff004109370 0x7ff0041a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0041089b0 0x7ff00419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 --2- 172.21.15.6:0/3254089467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0041077c0 0x7ff00419d860 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.192+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 >> 172.21.15.6:0/3254089467 conn(0x7ff004076780 msgr2=0x7ff00410bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.193+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 shutdown_connections 2024-02-26T06:24:56.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:56.193+0000 7ff009600640 1 -- 172.21.15.6:0/3254089467 wait complete. 2024-02-26T06:24:56.697 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:56 smithi090 systemd[1]: Starting Ceph mds.h for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:56.991 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:56 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3254089467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:56.992 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:56 smithi090 podman[67003]: 2024-02-26 06:24:56.688945639 +0000 UTC m=+0.017799342 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:56.992 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:56 smithi090 podman[67003]: 2024-02-26 06:24:56.790163602 +0000 UTC m=+0.119017308 container create a4ba64c7d6e944a76f5a39b8527d482aa99f763a781b30f67edf607b25d86f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:24:56.992 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:56 smithi090 podman[67003]: 2024-02-26 06:24:56.992781297 +0000 UTC m=+0.321634980 container init a4ba64c7d6e944a76f5a39b8527d482aa99f763a781b30f67edf607b25d86f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:24:56.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:56 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3254089467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:57.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:56 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3254089467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:57.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:56 smithi090 podman[67003]: 2024-02-26 06:24:56.995337483 +0000 UTC m=+0.324191177 container start a4ba64c7d6e944a76f5a39b8527d482aa99f763a781b30f67edf607b25d86f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-02-26T06:24:57.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mds[67023]: -- 172.21.15.90:0/694936187 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5618b0a441a0 con 0x5618b0a1a800 2024-02-26T06:24:57.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h[67019]: starting mds.h at 2024-02-26T06:24:57.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:57 smithi090 bash[67003]: a4ba64c7d6e944a76f5a39b8527d482aa99f763a781b30f67edf607b25d86f0e 2024-02-26T06:24:57.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:24:57 smithi090 systemd[1]: Started Ceph mds.h for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:57 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:58.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:58.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:57 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:24:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:24:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:24:58.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:57 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:24:58.214 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:24:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 -- 172.21.15.6:0/439239303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 msgr2=0x7fae3c104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 --2- 172.21.15.6:0/439239303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 0x7fae3c104930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae24009900 tx=0x7fae2402bc30 comp rx=0 tx=0).stop 2024-02-26T06:24:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 -- 172.21.15.6:0/439239303 shutdown_connections 2024-02-26T06:24:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 --2- 172.21.15.6:0/439239303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c10b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 --2- 172.21.15.6:0/439239303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 0x7fae3c104930 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 --2- 172.21.15.6:0/439239303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 0x7fae3c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 -- 172.21.15.6:0/439239303 >> 172.21.15.6:0/439239303 conn(0x7fae3c0fe850 msgr2=0x7fae3c100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:58.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.448+0000 7fae41400640 1 -- 172.21.15.6:0/439239303 shutdown_connections 2024-02-26T06:24:58.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.449+0000 7fae41400640 1 -- 172.21.15.6:0/439239303 wait complete. 2024-02-26T06:24:58.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.450+0000 7fae41400640 1 Processor -- start 2024-02-26T06:24:58.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.450+0000 7fae41400640 1 -- start start 2024-02-26T06:24:58.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.450+0000 7fae41400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 0x7fae3c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.450+0000 7fae41400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 0x7fae3c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.450+0000 7fae41400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.450+0000 7fae41400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c1a0e20 con 0x7fae3c104e70 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae41400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c1a0f90 con 0x7fae3c1032c0 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae41400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c1a1150 con 0x7fae3c1044b0 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c1a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56068/0 (socket says 172.21.15.6:56068) 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 -- 172.21.15.6:0/662351610 learned_addr learned my addr 172.21.15.6:0/662351610 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae3be00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 0x7fae3c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae33e00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 0x7fae3c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 -- 172.21.15.6:0/662351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 msgr2=0x7fae3c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:58.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 0x7fae3c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 -- 172.21.15.6:0/662351610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 msgr2=0x7fae3c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 0x7fae3c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae40a00640 1 -- 172.21.15.6:0/662351610 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009490 con 0x7fae3c104e70 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.451+0000 7fae3be00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 0x7fae3c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.452+0000 7fae40a00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c1a08e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae3400eb10 tx=0x7fae3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.452+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34004b20 con 0x7fae3c104e70 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.452+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae34004c80 con 0x7fae3c104e70 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.452+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34010650 con 0x7fae3c104e70 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.452+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24009560 con 0x7fae3c104e70 2024-02-26T06:24:58.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.452+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1a1730 con 0x7fae3c104e70 2024-02-26T06:24:58.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.453+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c10a580 con 0x7fae3c104e70 2024-02-26T06:24:58.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.455+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fae34005020 con 0x7fae3c104e70 2024-02-26T06:24:58.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.458+0000 7fae39600640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae200a2130 0x7fae200a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:24:58.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.458+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fae34014070 con 0x7fae3c104e70 2024-02-26T06:24:58.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.458+0000 7fae3be00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae200a2130 0x7fae200a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:24:58.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.459+0000 7fae3be00640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae200a2130 0x7fae200a45f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae3c104310 tx=0x7fae2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:24:58.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.462+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae340971d0 con 0x7fae3c104e70 2024-02-26T06:24:58.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.705+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae3c10ecd0 con 0x7fae3c104e70 2024-02-26T06:24:58.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.707+0000 7fae39600640 1 -- 172.21.15.6:0/662351610 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6844 (secure 0 0 0) 0x7fae3409a890 con 0x7fae3c104e70 2024-02-26T06:24:58.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:24:58.707 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T06:24:58.708 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae200a2130 msgr2=0x7fae200a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae200a2130 0x7fae200a45f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae3c104310 tx=0x7fae2c005f90 comp rx=0 tx=0).stop 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 msgr2=0x7fae3c1a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c1a08e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae3400eb10 tx=0x7fae3400c2d0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 shutdown_connections 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae3c104e70 0x7fae3c1a08e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae3c1044b0 0x7fae3c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae200a2130 0x7fae200a45f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 --2- 172.21.15.6:0/662351610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c1032c0 0x7fae3c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:24:58.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 >> 172.21.15.6:0/662351610 conn(0x7fae3c0fe850 msgr2=0x7fae3c107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:24:58.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 shutdown_connections 2024-02-26T06:24:58.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:24:58.710+0000 7fae41400640 1 -- 172.21.15.6:0/662351610 wait complete. 2024-02-26T06:24:58.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:58 smithi167 ceph-mon[36229]: Deploying daemon mds.i on smithi167 2024-02-26T06:24:58.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:58 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] up:boot 2024-02-26T06:24:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:58 smithi167 ceph-mon[36229]: fsmap 8 up:standby 2024-02-26T06:24:58.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:58 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:24:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:58 smithi090 ceph-mon[35681]: Deploying daemon mds.i on smithi167 2024-02-26T06:24:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:58 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] up:boot 2024-02-26T06:24:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:58 smithi090 ceph-mon[35681]: fsmap 8 up:standby 2024-02-26T06:24:59.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:58 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:24:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:58 smithi006 ceph-mon[31790]: Deploying daemon mds.i on smithi167 2024-02-26T06:24:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:58 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] up:boot 2024-02-26T06:24:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:58 smithi006 ceph-mon[31790]: fsmap 8 up:standby 2024-02-26T06:24:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:58 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:24:59.495 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 systemd[1]: Starting Ceph mds.i for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:59 smithi167 ceph-mon[36229]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:24:59 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/662351610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 podman[67760]: 2024-02-26 06:24:59.486252025 +0000 UTC m=+0.018386114 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 podman[67760]: 2024-02-26 06:24:59.599912258 +0000 UTC m=+0.132046329 container create 688601f53c498f9655e774b3eae69dacdd2500e1c961c7002109c6d51cdc41b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 podman[67760]: 2024-02-26 06:24:59.819341843 +0000 UTC m=+0.351475927 container init 688601f53c498f9655e774b3eae69dacdd2500e1c961c7002109c6d51cdc41b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 podman[67760]: 2024-02-26 06:24:59.821956763 +0000 UTC m=+0.354090840 container start 688601f53c498f9655e774b3eae69dacdd2500e1c961c7002109c6d51cdc41b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-26T06:24:59.844 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 ceph-mds[67779]: -- 172.21.15.167:0/3617569547 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5599d2fb21a0 con 0x5599d2f89000 2024-02-26T06:25:00.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:59 smithi090 ceph-mon[35681]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:00.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:24:59 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/662351610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:00.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:59 smithi006 ceph-mon[31790]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:00.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:24:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/662351610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:00.244 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: starting mds.i at 2024-02-26T06:25:00.244 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 bash[67760]: 688601f53c498f9655e774b3eae69dacdd2500e1c961c7002109c6d51cdc41b2 2024-02-26T06:25:00.244 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:24:59 smithi167 systemd[1]: Started Ceph mds.i for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:25:00.731 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:00.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 -- 172.21.15.6:0/2552604939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 msgr2=0x7f9ee4103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 --2- 172.21.15.6:0/2552604939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4103860 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 -- 172.21.15.6:0/2552604939 shutdown_connections 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 --2- 172.21.15.6:0/2552604939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee4104ff0 0x7f9ee410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 --2- 172.21.15.6:0/2552604939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee4104630 0x7f9ee4104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 --2- 172.21.15.6:0/2552604939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4103860 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 -- 172.21.15.6:0/2552604939 >> 172.21.15.6:0/2552604939 conn(0x7f9ee40fe990 msgr2=0x7f9ee4100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 -- 172.21.15.6:0/2552604939 shutdown_connections 2024-02-26T06:25:00.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.916+0000 7f9eeb800640 1 -- 172.21.15.6:0/2552604939 wait complete. 2024-02-26T06:25:00.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 Processor -- start 2024-02-26T06:25:00.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 -- start start 2024-02-26T06:25:00.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:00.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee4104630 0x7f9ee41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:00.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee4104ff0 0x7f9ee41a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41a0f60 con 0x7f9ee4103440 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41a10d0 con 0x7f9ee4104ff0 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.917+0000 7f9eeb800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41a1290 con 0x7f9ee4104630 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee4104630 0x7f9ee41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56072/0 (socket says 172.21.15.6:56072) 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee4104ff0 0x7f9ee41a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 -- 172.21.15.6:0/1747353584 learned_addr learned my addr 172.21.15.6:0/1747353584 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 -- 172.21.15.6:0/1747353584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee4104630 msgr2=0x7f9ee41999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee4104630 0x7f9ee41999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 -- 172.21.15.6:0/1747353584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee4104ff0 msgr2=0x7f9ee41a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee4104ff0 0x7f9ee41a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.918+0000 7f9ee8e00640 1 -- 172.21.15.6:0/1747353584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009560 con 0x7f9ee4103440 2024-02-26T06:25:00.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.919+0000 7f9ee8e00640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4199470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005ab0 tx=0x7f9ed8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:00.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.919+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80043d0 con 0x7f9ee4103440 2024-02-26T06:25:00.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.919+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee41a1510 con 0x7f9ee4103440 2024-02-26T06:25:00.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.919+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed80052d0 con 0x7f9ee4103440 2024-02-26T06:25:00.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.919+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8003d40 con 0x7f9ee4103440 2024-02-26T06:25:00.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.919+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee41a1a20 con 0x7f9ee4103440 2024-02-26T06:25:00.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.920+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee410a760 con 0x7f9ee4103440 2024-02-26T06:25:00.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.922+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9ed8005440 con 0x7f9ee4103440 2024-02-26T06:25:00.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.923+0000 7f9ee1600640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9ecc0a20c0 0x7f9ecc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:00.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.924+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ed80ef9c0 con 0x7f9ee4103440 2024-02-26T06:25:00.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.924+0000 7f9ee3e00640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9ecc0a20c0 0x7f9ecc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:00.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.924+0000 7f9ee3e00640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9ecc0a20c0 0x7f9ecc0a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ee419a980 tx=0x7f9ed0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:00.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:00.928+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ed80b8d90 con 0x7f9ee4103440 2024-02-26T06:25:01.116 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.116 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:boot 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: fsmap 9 up:standby 2024-02-26T06:25:01.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:00 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:25:01.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.174+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ee402cdd0 con 0x7f9ee4103440 2024-02-26T06:25:01.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.176+0000 7f9ee1600640 1 -- 172.21.15.6:0/1747353584 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7640 (secure 0 0 0) 0x7f9ed80bc450 con 0x7f9ee4103440 2024-02-26T06:25:01.176 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:01.176 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:25:01.177 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 10 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.178+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9ecc0a20c0 msgr2=0x7f9ecc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.178+0000 7f9eeb800640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9ecc0a20c0 0x7f9ecc0a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ee419a980 tx=0x7f9ed0005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.178+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 msgr2=0x7f9ee4199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.178+0000 7f9eeb800640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4199470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005ab0 tx=0x7f9ed8005e50 comp rx=0 tx=0).stop 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 shutdown_connections 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9ecc0a20c0 0x7f9ecc0a4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee4104ff0 0x7f9ee41a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee4104630 0x7f9ee41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 --2- 172.21.15.6:0/1747353584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ee4103440 0x7f9ee4199470 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 >> 172.21.15.6:0/1747353584 conn(0x7f9ee40fe990 msgr2=0x7f9ee4107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 shutdown_connections 2024-02-26T06:25:01.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:01.179+0000 7f9eeb800640 1 -- 172.21.15.6:0/1747353584 wait complete. 2024-02-26T06:25:01.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:boot 2024-02-26T06:25:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: fsmap 9 up:standby 2024-02-26T06:25:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:00 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:boot 2024-02-26T06:25:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: fsmap 9 up:standby 2024-02-26T06:25:01.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:00 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:25:02.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:01 smithi167 ceph-mon[36229]: Deploying daemon mds.g on smithi006 2024-02-26T06:25:02.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:01 smithi167 ceph-mon[36229]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:01 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1747353584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:01 smithi090 ceph-mon[35681]: Deploying daemon mds.g on smithi006 2024-02-26T06:25:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:01 smithi090 ceph-mon[35681]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:01 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1747353584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:01 smithi006 ceph-mon[31790]: Deploying daemon mds.g on smithi006 2024-02-26T06:25:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:01 smithi006 ceph-mon[31790]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1747353584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:03.200 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:03.256 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 systemd[1]: Starting Ceph mds.g for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:25:03.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 -- 172.21.15.6:0/2801846042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8075b40 msgr2=0x7fafd8075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 --2- 172.21.15.6:0/2801846042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8075b40 0x7fafd8075f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009900 tx=0x7fafc802bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 -- 172.21.15.6:0/2801846042 shutdown_connections 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 --2- 172.21.15.6:0/2801846042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8077720 0x7fafd8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 --2- 172.21.15.6:0/2801846042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 0x7fafd80771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 --2- 172.21.15.6:0/2801846042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8075b40 0x7fafd8075f60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 -- 172.21.15.6:0/2801846042 >> 172.21.15.6:0/2801846042 conn(0x7fafd80fe9e0 msgr2=0x7fafd8100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 -- 172.21.15.6:0/2801846042 shutdown_connections 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.416+0000 7fafdca00640 1 -- 172.21.15.6:0/2801846042 wait complete. 2024-02-26T06:25:03.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.417+0000 7fafdca00640 1 Processor -- start 2024-02-26T06:25:03.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 -- start start 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8075b40 0x7fafd819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 0x7fafd819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8077720 0x7fafd81a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81a5340 con 0x7fafd8077720 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81a54b0 con 0x7fafd8075b40 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafdca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81a5670 con 0x7fafd8076d80 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.418+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 0x7fafd819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:03.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8075b40 0x7fafd819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 0x7fafd819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:55712/0 (socket says 172.21.15.6:55712) 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafcfe00640 1 -- 172.21.15.6:0/3758601864 learned_addr learned my addr 172.21.15.6:0/3758601864 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd6800640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8077720 0x7fafd81a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 -- 172.21.15.6:0/3758601864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 msgr2=0x7fafd819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 0x7fafd819dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 -- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8077720 msgr2=0x7fafd81a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8077720 0x7fafd81a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 -- 172.21.15.6:0/3758601864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8009560 con 0x7fafd8075b40 2024-02-26T06:25:03.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.419+0000 7fafd5e00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8075b40 0x7fafd819d850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafc8000c00 tx=0x7fafc8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:03.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.420+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8004aa0 con 0x7fafd8075b40 2024-02-26T06:25:03.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.420+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafc80052d0 con 0x7fafd8075b40 2024-02-26T06:25:03.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.420+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8003e80 con 0x7fafd8075b40 2024-02-26T06:25:03.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.420+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd81a58f0 con 0x7fafd8075b40 2024-02-26T06:25:03.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.420+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd81a5d80 con 0x7fafd8075b40 2024-02-26T06:25:03.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.422+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd8068450 con 0x7fafd8075b40 2024-02-26T06:25:03.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.424+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fafc8005000 con 0x7fafd8075b40 2024-02-26T06:25:03.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.426+0000 7fafcea00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fafbc0a2140 0x7fafbc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:03.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.426+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fafc80f0210 con 0x7fafd8075b40 2024-02-26T06:25:03.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.426+0000 7fafcfe00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fafbc0a2140 0x7fafbc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:03.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.427+0000 7fafcfe00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fafbc0a2140 0x7fafbc0a4600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafd819ed60 tx=0x7fafc000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.427+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafc80b95e0 con 0x7fafd8075b40 2024-02-26T06:25:03.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.680+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafd810ff80 con 0x7fafd8075b40 2024-02-26T06:25:03.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.681+0000 7fafcea00640 1 -- 172.21.15.6:0/3758601864 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7640 (secure 0 0 0) 0x7fafc80bcca0 con 0x7fafd8075b40 2024-02-26T06:25:03.682 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:03.682 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:25:03.682 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 10 2024-02-26T06:25:03.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fafbc0a2140 msgr2=0x7fafbc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:03.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fafbc0a2140 0x7fafbc0a4600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafd819ed60 tx=0x7fafc000b4a0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8075b40 msgr2=0x7fafd819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8075b40 0x7fafd819d850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafc8000c00 tx=0x7fafc8004270 comp rx=0 tx=0).stop 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 shutdown_connections 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd8077720 0x7fafd81a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd8076d80 0x7fafd819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fafbc0a2140 0x7fafbc0a4600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 --2- 172.21.15.6:0/3758601864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8075b40 0x7fafd819d850 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 >> 172.21.15.6:0/3758601864 conn(0x7fafd80fe9e0 msgr2=0x7fafd8100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 shutdown_connections 2024-02-26T06:25:03.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:03.684+0000 7fafdca00640 1 -- 172.21.15.6:0/3758601864 wait complete. 2024-02-26T06:25:03.908 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 podman[73119]: 2024-02-26 06:25:03.527302923 +0000 UTC m=+0.035179460 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:25:03.908 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 podman[73119]: 2024-02-26 06:25:03.640961258 +0000 UTC m=+0.148837839 container create e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-26T06:25:03.908 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 podman[73119]: 2024-02-26 06:25:03.907298946 +0000 UTC m=+0.415175574 container init e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:25:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:03 smithi006 ceph-mon[31790]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:04.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:03 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3758601864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:04.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 podman[73119]: 2024-02-26 06:25:03.91824403 +0000 UTC m=+0.426120567 container start e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T06:25:04.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 ceph-mds[73140]: -- 172.21.15.6:0/4237941962 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557058b7a1a0 con 0x557058b50c00 2024-02-26T06:25:04.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g[73136]: starting mds.g at 2024-02-26T06:25:04.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:04 smithi006 bash[73119]: e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 2024-02-26T06:25:04.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:25:04 smithi006 systemd[1]: Started Ceph mds.g for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:25:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:03 smithi090 ceph-mon[35681]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:03 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3758601864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:04.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:03 smithi167 ceph-mon[36229]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:04.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:03 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3758601864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:05.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6838/1183641688,v1:172.21.15.6:6839/1183641688] up:boot 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: fsmap 10 up:standby 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:05.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:05 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6838/1183641688,v1:172.21.15.6:6839/1183641688] up:boot 2024-02-26T06:25:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: fsmap 10 up:standby 2024-02-26T06:25:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:25:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:05.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:05.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:05 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6838/1183641688,v1:172.21.15.6:6839/1183641688] up:boot 2024-02-26T06:25:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: fsmap 10 up:standby 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:05.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:05 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:05.705 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.912+0000 7f6bc9800640 1 -- 172.21.15.6:0/56427218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 msgr2=0x7f6bc4104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.912+0000 7f6bc9800640 1 --2- 172.21.15.6:0/56427218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4104930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb002bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 -- 172.21.15.6:0/56427218 shutdown_connections 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 --2- 172.21.15.6:0/56427218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc4104e70 0x7f6bc410b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 --2- 172.21.15.6:0/56427218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4104930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 --2- 172.21.15.6:0/56427218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bc41032c0 0x7f6bc41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 -- 172.21.15.6:0/56427218 >> 172.21.15.6:0/56427218 conn(0x7f6bc40fe850 msgr2=0x7f6bc4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:05.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 -- 172.21.15.6:0/56427218 shutdown_connections 2024-02-26T06:25:05.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.913+0000 7f6bc9800640 1 -- 172.21.15.6:0/56427218 wait complete. 2024-02-26T06:25:05.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 Processor -- start 2024-02-26T06:25:05.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 -- start start 2024-02-26T06:25:05.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc41032c0 0x7f6bc4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:05.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:05.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bc4104e70 0x7f6bc41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc41a0e00 con 0x7f6bc41044b0 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc41a0f70 con 0x7f6bc41032c0 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc41a1130 con 0x7f6bc4104e70 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc3400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc3400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56132/0 (socket says 172.21.15.6:56132) 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc3400640 1 -- 172.21.15.6:0/1559607290 learned_addr learned my addr 172.21.15.6:0/1559607290 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.914+0000 7f6bc3e00640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc41032c0 0x7f6bc4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc8e00640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bc4104e70 0x7f6bc41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3e00640 1 -- 172.21.15.6:0/1559607290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bc4104e70 msgr2=0x7f6bc41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3e00640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bc4104e70 0x7f6bc41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3e00640 1 -- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 msgr2=0x7f6bc4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3e00640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3e00640 1 -- 172.21.15.6:0/1559607290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0009560 con 0x7f6bc41032c0 2024-02-26T06:25:05.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3400640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:05.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.915+0000 7f6bc3e00640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc41032c0 0x7f6bc4199310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400e9c0 tx=0x7f6bb400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:05.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.916+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400ccc0 con 0x7f6bc41032c0 2024-02-26T06:25:05.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.916+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bb400ce20 con 0x7f6bc41032c0 2024-02-26T06:25:05.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.916+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4010430 con 0x7f6bc41032c0 2024-02-26T06:25:05.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.916+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc41a1410 con 0x7f6bc41032c0 2024-02-26T06:25:05.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.916+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc41a18d0 con 0x7f6bc41032c0 2024-02-26T06:25:05.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.917+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc410a5e0 con 0x7f6bc41032c0 2024-02-26T06:25:05.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.919+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f6bb4003f60 con 0x7f6bc41032c0 2024-02-26T06:25:05.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.920+0000 7f6bc0c00640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ba80a20b0 0x7f6ba80a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:05.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.921+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6bb4014070 con 0x7f6bc41032c0 2024-02-26T06:25:05.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.921+0000 7f6bc3400640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ba80a20b0 0x7f6ba80a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:05.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.922+0000 7f6bc3400640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ba80a20b0 0x7f6ba80a4570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6bc419a820 tx=0x7f6bb002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:05.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:05.922+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6bb4097110 con 0x7f6bc41032c0 2024-02-26T06:25:06.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.168+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bc410ecd0 con 0x7f6bc41032c0 2024-02-26T06:25:06.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.169+0000 7f6bc0c00640 1 -- 172.21.15.6:0/1559607290 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8429 (secure 0 0 0) 0x7f6bb409a7d0 con 0x7f6bc41032c0 2024-02-26T06:25:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:06.170 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:25:06.170 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 11 2024-02-26T06:25:06.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ba80a20b0 msgr2=0x7f6ba80a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ba80a20b0 0x7f6ba80a4570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6bc419a820 tx=0x7f6bb002c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc41032c0 msgr2=0x7f6bc4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc41032c0 0x7f6bc4199310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400e9c0 tx=0x7f6bb400ee80 comp rx=0 tx=0).stop 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 shutdown_connections 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6bc4104e70 0x7f6bc41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc41044b0 0x7f6bc4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ba80a20b0 0x7f6ba80a4570 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 --2- 172.21.15.6:0/1559607290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc41032c0 0x7f6bc4199310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 >> 172.21.15.6:0/1559607290 conn(0x7f6bc40fe850 msgr2=0x7f6bc41077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 shutdown_connections 2024-02-26T06:25:06.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:06.172+0000 7f6bc9800640 1 -- 172.21.15.6:0/1559607290 wait complete. 2024-02-26T06:25:06.329 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:06 smithi006 ceph-mon[31790]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:06.329 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:06 smithi006 ceph-mon[31790]: Deploying daemon mds.a on smithi006 2024-02-26T06:25:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:06 smithi090 ceph-mon[35681]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:06 smithi090 ceph-mon[35681]: Deploying daemon mds.a on smithi006 2024-02-26T06:25:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:06 smithi167 ceph-mon[36229]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:06 smithi167 ceph-mon[36229]: Deploying daemon mds.a on smithi006 2024-02-26T06:25:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:07 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1559607290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:07.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:07 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1559607290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:07.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:07 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1559607290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:07.876 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:07 smithi006 systemd[1]: Starting Ceph mds.a for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:25:08.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:08 smithi006 ceph-mon[31790]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:08.129 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:07 smithi006 podman[73462]: 2024-02-26 06:25:07.780476555 +0000 UTC m=+0.032630576 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:25:08.130 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:07 smithi006 podman[73462]: 2024-02-26 06:25:07.948774759 +0000 UTC m=+0.200928808 container create 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-02-26T06:25:08.192 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:08.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:08 smithi090 ceph-mon[35681]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 -- 172.21.15.6:0/2931088571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc077720 msgr2=0x7ffabc077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 --2- 172.21.15.6:0/2931088571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc077720 0x7ffabc077fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009900 tx=0x7ffab402bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 -- 172.21.15.6:0/2931088571 shutdown_connections 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 --2- 172.21.15.6:0/2931088571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc077720 0x7ffabc077fc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 --2- 172.21.15.6:0/2931088571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffabc076d80 0x7ffabc0771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 --2- 172.21.15.6:0/2931088571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc075b40 0x7ffabc075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 -- 172.21.15.6:0/2931088571 >> 172.21.15.6:0/2931088571 conn(0x7ffabc0fe9e0 msgr2=0x7ffabc100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 -- 172.21.15.6:0/2931088571 shutdown_connections 2024-02-26T06:25:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.415+0000 7ffac1600640 1 -- 172.21.15.6:0/2931088571 wait complete. 2024-02-26T06:25:08.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.416+0000 7ffac1600640 1 Processor -- start 2024-02-26T06:25:08.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 -- start start 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffabc075b40 0x7ffabc19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 0x7ffabc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc077720 0x7ffabc1a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a52f0 con 0x7ffabc075b40 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a5460 con 0x7ffabc076d80 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffac1600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a5620 con 0x7ffabc077720 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffabaa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffabc075b40 0x7ffabc19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffaba000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 0x7ffabc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffaba000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 0x7ffabc19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34904/0 (socket says 172.21.15.6:34904) 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffaba000640 1 -- 172.21.15.6:0/3829816484 learned_addr learned my addr 172.21.15.6:0/3829816484 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.417+0000 7ffabb400640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc077720 0x7ffabc1a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffaba000640 1 -- 172.21.15.6:0/3829816484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc077720 msgr2=0x7ffabc1a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffaba000640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc077720 0x7ffabc1a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffaba000640 1 -- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffabc075b40 msgr2=0x7ffabc19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffaba000640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffabc075b40 0x7ffabc19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffaba000640 1 -- 172.21.15.6:0/3829816484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab4009560 con 0x7ffabc076d80 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffaba000640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 0x7ffabc19dd40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00db80 tx=0x7ffaac00b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac003b70 con 0x7ffabc076d80 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaac00de80 con 0x7ffabc076d80 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0106c0 con 0x7ffabc076d80 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.418+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1a5900 con 0x7ffabc076d80 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.419+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1a5dc0 con 0x7ffabc076d80 2024-02-26T06:25:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.420+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc10fee0 con 0x7ffabc076d80 2024-02-26T06:25:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.422+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ffaac004020 con 0x7ffabc076d80 2024-02-26T06:25:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.423+0000 7ffab3400640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffaa00a20c0 0x7ffaa00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.423+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffaac0cd870 con 0x7ffabc076d80 2024-02-26T06:25:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.423+0000 7ffabaa00640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffaa00a20c0 0x7ffaa00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:08.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.424+0000 7ffabaa00640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffaa00a20c0 0x7ffaa00a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009bd0 tx=0x7ffaa4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:08.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.428+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffaac096c40 con 0x7ffabc076d80 2024-02-26T06:25:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:08 smithi167 ceph-mon[36229]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:08.672 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:08 smithi006 podman[73462]: 2024-02-26 06:25:08.223640961 +0000 UTC m=+0.475794984 container init 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T06:25:08.672 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:08 smithi006 podman[73462]: 2024-02-26 06:25:08.235586504 +0000 UTC m=+0.487740524 container start 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-26T06:25:08.672 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:08 smithi006 ceph-mds[73494]: -- 172.21.15.6:0/1629635788 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5610274641a0 con 0x56102743b000 2024-02-26T06:25:08.672 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a[73477]: starting mds.a at 2024-02-26T06:25:08.672 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:08 smithi006 bash[73462]: 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e 2024-02-26T06:25:08.672 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:25:08 smithi006 systemd[1]: Started Ceph mds.a for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:25:08.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.672+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffabc02ce40 con 0x7ffabc076d80 2024-02-26T06:25:08.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.674+0000 7ffab3400640 1 -- 172.21.15.6:0/3829816484 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8429 (secure 0 0 0) 0x7ffaac00be60 con 0x7ffabc076d80 2024-02-26T06:25:08.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:08.674 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T06:25:08.675 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 11 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffaa00a20c0 msgr2=0x7ffaa00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffaa00a20c0 0x7ffaa00a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009bd0 tx=0x7ffaa4005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 msgr2=0x7ffabc19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 0x7ffabc19dd40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00db80 tx=0x7ffaac00b630 comp rx=0 tx=0).stop 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 shutdown_connections 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc077720 0x7ffabc1a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffaa00a20c0 0x7ffaa00a4580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc076d80 0x7ffabc19dd40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 --2- 172.21.15.6:0/3829816484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffabc075b40 0x7ffabc19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:08.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 >> 172.21.15.6:0/3829816484 conn(0x7ffabc0fe9e0 msgr2=0x7ffabc100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:08.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 shutdown_connections 2024-02-26T06:25:08.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:08.677+0000 7ffac1600640 1 -- 172.21.15.6:0/3829816484 wait complete. 2024-02-26T06:25:09.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:09.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:09.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: Deploying daemon mds.b on smithi090 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3829816484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] up:boot 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: fsmap 11 up:standby 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:09 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:09.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: Deploying daemon mds.b on smithi090 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3829816484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] up:boot 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: fsmap 11 up:standby 2024-02-26T06:25:09.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:09 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: Deploying daemon mds.b on smithi090 2024-02-26T06:25:09.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3829816484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:09.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] up:boot 2024-02-26T06:25:09.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: fsmap 11 up:standby 2024-02-26T06:25:09.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:09 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:25:10.696 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:10.815 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:10 smithi090 systemd[1]: Starting Ceph mds.b for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 -- 172.21.15.6:0/2385237318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 msgr2=0x7fd924075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 --2- 172.21.15.6:0/2385237318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 0x7fd924075f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd914009900 tx=0x7fd91402bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 -- 172.21.15.6:0/2385237318 shutdown_connections 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 --2- 172.21.15.6:0/2385237318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924077720 0x7fd924077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 --2- 172.21.15.6:0/2385237318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd924076d80 0x7fd9240771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 --2- 172.21.15.6:0/2385237318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 0x7fd924075f60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.902+0000 7fd929a00640 1 -- 172.21.15.6:0/2385237318 >> 172.21.15.6:0/2385237318 conn(0x7fd9240fe9e0 msgr2=0x7fd924100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.903+0000 7fd929a00640 1 -- 172.21.15.6:0/2385237318 shutdown_connections 2024-02-26T06:25:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.903+0000 7fd929a00640 1 -- 172.21.15.6:0/2385237318 wait complete. 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 Processor -- start 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 -- start start 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 0x7fd92419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 0x7fd92419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd924077720 0x7fd9241a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a5360 con 0x7fd924075b40 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a54d0 con 0x7fd924076d80 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd929a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a5690 con 0x7fd924077720 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd922000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 0x7fd92419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd922000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 0x7fd92419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34934/0 (socket says 172.21.15.6:34934) 2024-02-26T06:25:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd922000640 1 -- 172.21.15.6:0/3961950774 learned_addr learned my addr 172.21.15.6:0/3961950774 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd922a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 0x7fd92419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.904+0000 7fd923400640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd924077720 0x7fd9241a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd922000640 1 -- 172.21.15.6:0/3961950774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd924077720 msgr2=0x7fd9241a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd922000640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd924077720 0x7fd9241a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd922000640 1 -- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 msgr2=0x7fd92419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd922000640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 0x7fd92419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd922000640 1 -- 172.21.15.6:0/3961950774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009490 con 0x7fd924076d80 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd922000640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 0x7fd92419ddb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd91000ea60 tx=0x7fd91000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000cde0 con 0x7fd924076d80 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd914009560 con 0x7fd924076d80 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.905+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9100035f0 con 0x7fd924076d80 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.906+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910010690 con 0x7fd924076d80 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.906+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241a5d40 con 0x7fd924076d80 2024-02-26T06:25:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.907+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92410ff30 con 0x7fd924076d80 2024-02-26T06:25:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.909+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd910003af0 con 0x7fd924076d80 2024-02-26T06:25:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.910+0000 7fd91b400640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd9080a2140 0x7fd9080a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.910+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd910014070 con 0x7fd924076d80 2024-02-26T06:25:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.910+0000 7fd922a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd9080a2140 0x7fd9080a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:10.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.911+0000 7fd922a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd9080a2140 0x7fd9080a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd914006010 tx=0x7fd914000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:10.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:10.914+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd910097110 con 0x7fd924076d80 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:10 smithi090 podman[67320]: 2024-02-26 06:25:10.815693109 +0000 UTC m=+0.103418852 container create 9a03e2fa1b16d03184bbb396797b6f489931661a9e1d1d51e07909224d97e155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:10 smithi090 podman[67320]: 2024-02-26 06:25:10.729910892 +0000 UTC m=+0.017636658 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:10 smithi090 podman[67320]: 2024-02-26 06:25:10.993310269 +0000 UTC m=+0.281036032 container init 9a03e2fa1b16d03184bbb396797b6f489931661a9e1d1d51e07909224d97e155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:10 smithi090 podman[67320]: 2024-02-26 06:25:10.996063351 +0000 UTC m=+0.283789104 container start 9a03e2fa1b16d03184bbb396797b6f489931661a9e1d1d51e07909224d97e155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mds[67339]: -- 172.21.15.90:0/3243580653 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56428d0061a0 con 0x56428cfdc800 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: starting mds.b at 2024-02-26T06:25:11.068 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:11 smithi090 bash[67320]: 9a03e2fa1b16d03184bbb396797b6f489931661a9e1d1d51e07909224d97e155 2024-02-26T06:25:11.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.159+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9241131d0 con 0x7fd924076d80 2024-02-26T06:25:11.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.160+0000 7fd91b400640 1 -- 172.21.15.6:0/3961950774 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9218 (secure 0 0 0) 0x7fd91009a7d0 con 0x7fd924076d80 2024-02-26T06:25:11.160 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:11.160 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-02-26T06:25:11.160 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 12 2024-02-26T06:25:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd9080a2140 msgr2=0x7fd9080a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd9080a2140 0x7fd9080a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd914006010 tx=0x7fd914000bc0 comp rx=0 tx=0).stop 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 msgr2=0x7fd92419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 0x7fd92419ddb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd91000ea60 tx=0x7fd91000ef20 comp rx=0 tx=0).stop 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 shutdown_connections 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd924077720 0x7fd9241a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd9080a2140 0x7fd9080a4600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd924076d80 0x7fd92419ddb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 --2- 172.21.15.6:0/3961950774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd924075b40 0x7fd92419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 >> 172.21.15.6:0/3961950774 conn(0x7fd9240fe9e0 msgr2=0x7fd9241000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 shutdown_connections 2024-02-26T06:25:11.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:11.163+0000 7fd929a00640 1 -- 172.21.15.6:0/3961950774 wait complete. 2024-02-26T06:25:11.365 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:11 smithi090 systemd[1]: Started Ceph mds.b for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:25:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3961950774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:11.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:11 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3961950774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.852 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:11 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3961950774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:11 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:12.537 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:12 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] up:boot 2024-02-26T06:25:12.538 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:12 smithi090 ceph-mon[35681]: fsmap 12 up:standby 2024-02-26T06:25:12.538 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:25:12.538 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:12.538 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:12.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:12 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] up:boot 2024-02-26T06:25:12.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:12 smithi006 ceph-mon[31790]: fsmap 12 up:standby 2024-02-26T06:25:12.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:25:12.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:12.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:12 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] up:boot 2024-02-26T06:25:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:12 smithi167 ceph-mon[36229]: fsmap 12 up:standby 2024-02-26T06:25:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:25:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:12.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:13.183 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 -- 172.21.15.6:0/580166521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c101b10 msgr2=0x7fb36c103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 --2- 172.21.15.6:0/580166521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c101b10 0x7fb36c103f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009900 tx=0x7fb35c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 -- 172.21.15.6:0/580166521 shutdown_connections 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 --2- 172.21.15.6:0/580166521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb36c106d70 0x7fb36c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 --2- 172.21.15.6:0/580166521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36c104440 0x7fb36c106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 --2- 172.21.15.6:0/580166521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c101b10 0x7fb36c103f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.391+0000 7fb371600640 1 -- 172.21.15.6:0/580166521 >> 172.21.15.6:0/580166521 conn(0x7fb36c0fb500 msgr2=0x7fb36c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:13.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.392+0000 7fb371600640 1 -- 172.21.15.6:0/580166521 shutdown_connections 2024-02-26T06:25:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.392+0000 7fb371600640 1 -- 172.21.15.6:0/580166521 wait complete. 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 Processor -- start 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 -- start start 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb36c101b10 0x7fb36c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 0x7fb36c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36c106d70 0x7fb36c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1a0e70 con 0x7fb36c101b10 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1a0fe0 con 0x7fb36c104440 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb371600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1a11a0 con 0x7fb36c106d70 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb36b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 0x7fb36c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb36be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb36c101b10 0x7fb36c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb36b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 0x7fb36c1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34948/0 (socket says 172.21.15.6:34948) 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.393+0000 7fb36b400640 1 -- 172.21.15.6:0/3779915999 learned_addr learned my addr 172.21.15.6:0/3779915999 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb370c00640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36c106d70 0x7fb36c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb36b400640 1 -- 172.21.15.6:0/3779915999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36c106d70 msgr2=0x7fb36c1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb36b400640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36c106d70 0x7fb36c1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb36b400640 1 -- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb36c101b10 msgr2=0x7fb36c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb36b400640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb36c101b10 0x7fb36c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb36b400640 1 -- 172.21.15.6:0/3779915999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c009560 con 0x7fb36c104440 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb36b400640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 0x7fb36c1998c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb35800e9f0 tx=0x7fb35800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3580096c0 con 0x7fb36c104440 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb358005500 con 0x7fb36c104440 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.394+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c1a1480 con 0x7fb36c104440 2024-02-26T06:25:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.395+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358010430 con 0x7fb36c104440 2024-02-26T06:25:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.395+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c1a1990 con 0x7fb36c104440 2024-02-26T06:25:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.396+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c1088c0 con 0x7fb36c104440 2024-02-26T06:25:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.398+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb358005020 con 0x7fb36c104440 2024-02-26T06:25:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.399+0000 7fb368c00640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb3500a2130 0x7fb3500a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.399+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb358014070 con 0x7fb36c104440 2024-02-26T06:25:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.400+0000 7fb36be00640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb3500a2130 0x7fb3500a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.400+0000 7fb36be00640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb3500a2130 0x7fb3500a45f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb35c000c00 tx=0x7fb35c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:13.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.403+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb358096f30 con 0x7fb36c104440 2024-02-26T06:25:13.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.648+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb36c108330 con 0x7fb36c104440 2024-02-26T06:25:13.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.648+0000 7fb368c00640 1 -- 172.21.15.6:0/3779915999 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10007 (secure 0 0 0) 0x7fb35809a5f0 con 0x7fb36c104440 2024-02-26T06:25:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:13.648 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-26T06:25:13.648 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 13 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb3500a2130 msgr2=0x7fb3500a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb3500a2130 0x7fb3500a45f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb35c000c00 tx=0x7fb35c02c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 msgr2=0x7fb36c1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 0x7fb36c1998c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb35800e9f0 tx=0x7fb35800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 shutdown_connections 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36c106d70 0x7fb36c1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb3500a2130 0x7fb3500a45f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb36c104440 0x7fb36c1998c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 --2- 172.21.15.6:0/3779915999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb36c101b10 0x7fb36c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 >> 172.21.15.6:0/3779915999 conn(0x7fb36c0fb500 msgr2=0x7fb36c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 shutdown_connections 2024-02-26T06:25:13.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.651+0000 7fb371600640 1 -- 172.21.15.6:0/3779915999 wait complete. 2024-02-26T06:25:13.671 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:25:13.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1008823268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 msgr2=0x7f1bd41069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1008823268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 0x7f1bd41069c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009900 tx=0x7f1bc002bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:13.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1008823268 shutdown_connections 2024-02-26T06:25:13.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1008823268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4106f00 0x7f1bd4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1008823268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 0x7f1bd41069c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1008823268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1008823268 >> 172.21.15.6:0/1008823268 conn(0x7f1bd40fb6b0 msgr2=0x7f1bd40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:13.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1008823268 shutdown_connections 2024-02-26T06:25:13.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.865+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1008823268 wait complete. 2024-02-26T06:25:13.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.866+0000 7f1bdbc00640 1 Processor -- start 2024-02-26T06:25:13.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.866+0000 7f1bdbc00640 1 -- start start 2024-02-26T06:25:13.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bdbc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bdbc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 0x7f1bd40ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bdbc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bd4106f00 0x7f1bd4100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bdbc00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd41008f0 con 0x7f1bd4101ca0 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bdbc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd4100a60 con 0x7f1bd4106f00 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bdbc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd41a12f0 con 0x7f1bd41045d0 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bd9200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bd9200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52154/0 (socket says 172.21.15.6:52154) 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bd9200640 1 -- 172.21.15.6:0/1357461280 learned_addr learned my addr 172.21.15.6:0/1357461280 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bd3e00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 0x7f1bd40ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.867+0000 7f1bd9c00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bd4106f00 0x7f1bd4100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd9200640 1 -- 172.21.15.6:0/1357461280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 msgr2=0x7f1bd40ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd9200640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 0x7f1bd40ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd9200640 1 -- 172.21.15.6:0/1357461280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bd4106f00 msgr2=0x7f1bd4100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd9200640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bd4106f00 0x7f1bd4100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd9200640 1 -- 172.21.15.6:0/1357461280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0009560 con 0x7f1bd4101ca0 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd9200640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4101610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800ea20 tx=0x7f1bc800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8009720 con 0x7f1bd4101ca0 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd41a15a0 con 0x7f1bd4101ca0 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc800cd80 con 0x7f1bd4101ca0 2024-02-26T06:25:13.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8010490 con 0x7f1bd4101ca0 2024-02-26T06:25:13.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.868+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd41a1a80 con 0x7f1bd4101ca0 2024-02-26T06:25:13.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.870+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd40fd1a0 con 0x7f1bd4101ca0 2024-02-26T06:25:13.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.872+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1bc8005020 con 0x7f1bd4101ca0 2024-02-26T06:25:13.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.873+0000 7f1bd1600640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1bbc0a2140 0x7f1bbc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:13.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.873+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1bc8014070 con 0x7f1bd4101ca0 2024-02-26T06:25:13.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.874+0000 7f1bd3e00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1bbc0a2140 0x7f1bbc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:13.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.874+0000 7f1bd3e00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1bbc0a2140 0x7f1bbc0a4600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005ab0 tx=0x7f1bc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:13.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:13.878+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1bc8096fe0 con 0x7f1bd4101ca0 2024-02-26T06:25:14.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.089+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bd410cbc0 con 0x7f1bd4101ca0 2024-02-26T06:25:14.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.090+0000 7f1bd1600640 1 -- 172.21.15.6:0/1357461280 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f1bc809a6a0 con 0x7f1bd4101ca0 2024-02-26T06:25:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:14.090 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":81,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:24:16.991212+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:25:14.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1bbc0a2140 msgr2=0x7f1bbc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1bbc0a2140 0x7f1bbc0a4600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005ab0 tx=0x7f1bc0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 msgr2=0x7f1bd4101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4101610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800ea20 tx=0x7f1bc800ee20 comp rx=0 tx=0).stop 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 shutdown_connections 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1bbc0a2140 0x7f1bbc0a4600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bd4106f00 0x7f1bd4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bd41045d0 0x7f1bd40ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 --2- 172.21.15.6:0/1357461280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bd4101ca0 0x7f1bd4101610 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 >> 172.21.15.6:0/1357461280 conn(0x7f1bd40fb6b0 msgr2=0x7f1bd4102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 shutdown_connections 2024-02-26T06:25:14.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.093+0000 7f1bdbc00640 1 -- 172.21.15.6:0/1357461280 wait complete. 2024-02-26T06:25:14.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:25:14.115 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:25:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:13 smithi167 ceph-mon[36229]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:13 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3779915999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 -- 172.21.15.6:0/3354405337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6401000a0 msgr2=0x7ff6401004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 --2- 172.21.15.6:0/3354405337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6401000a0 0x7ff6401004c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff634009900 tx=0x7ff63402bc50 comp rx=0 tx=0).stop 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 -- 172.21.15.6:0/3354405337 shutdown_connections 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 --2- 172.21.15.6:0/3354405337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff640101c50 0x7ff6401084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 --2- 172.21.15.6:0/3354405337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff640101290 0x7ff640101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 --2- 172.21.15.6:0/3354405337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6401000a0 0x7ff6401004c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 -- 172.21.15.6:0/3354405337 >> 172.21.15.6:0/3354405337 conn(0x7ff6400fb610 msgr2=0x7ff6400fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 -- 172.21.15.6:0/3354405337 shutdown_connections 2024-02-26T06:25:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.331+0000 7ff647400640 1 -- 172.21.15.6:0/3354405337 wait complete. 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 Processor -- start 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 -- start start 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6401000a0 0x7ff640199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff640101290 0x7ff6401999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 0x7ff6401a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401a0f70 con 0x7ff640101c50 2024-02-26T06:25:14.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.332+0000 7ff647400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401a10e0 con 0x7ff640101290 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff647400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401a12a0 con 0x7ff6401000a0 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 0x7ff6401a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 0x7ff6401a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52176/0 (socket says 172.21.15.6:52176) 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 -- 172.21.15.6:0/268361300 learned_addr learned my addr 172.21.15.6:0/268361300 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff644a00640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6401000a0 0x7ff640199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff63fe00640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff640101290 0x7ff6401999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 -- 172.21.15.6:0/268361300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6401000a0 msgr2=0x7ff640199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6401000a0 0x7ff640199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 -- 172.21.15.6:0/268361300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff640101290 msgr2=0x7ff6401999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff640101290 0x7ff6401999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 -- 172.21.15.6:0/268361300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634009560 con 0x7ff640101c50 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.333+0000 7ff645400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 0x7ff6401a0a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff63800eaa0 tx=0x7ff63800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.334+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800cdb0 con 0x7ff640101c50 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.334+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff638005020 con 0x7ff640101c50 2024-02-26T06:25:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.334+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6380116b0 con 0x7ff640101c50 2024-02-26T06:25:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.334+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401a1580 con 0x7ff640101c50 2024-02-26T06:25:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.334+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401a1a40 con 0x7ff640101c50 2024-02-26T06:25:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.335+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64010cbb0 con 0x7ff640101c50 2024-02-26T06:25:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.337+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff638005520 con 0x7ff640101c50 2024-02-26T06:25:14.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.339+0000 7ff63d600640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff6280a2140 0x7ff6280a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.340+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6380cd9a0 con 0x7ff640101c50 2024-02-26T06:25:14.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.340+0000 7ff644a00640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff6280a2140 0x7ff6280a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.340+0000 7ff644a00640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff6280a2140 0x7ff6280a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff634005ab0 tx=0x7ff634005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:14.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.344+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff638096e20 con 0x7ff640101c50 2024-02-26T06:25:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:13 smithi090 ceph-mon[35681]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:13 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3779915999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:14.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:13 smithi006 ceph-mon[31790]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:14.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:14.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:13 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3779915999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:14.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.552+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6401a1cc0 con 0x7ff640101c50 2024-02-26T06:25:14.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.552+0000 7ff63d600640 1 -- 172.21.15.6:0/268361300 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff63809a4e0 con 0x7ff640101c50 2024-02-26T06:25:14.552 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff6280a2140 msgr2=0x7ff6280a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff6280a2140 0x7ff6280a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff634005ab0 tx=0x7ff634005fd0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 msgr2=0x7ff6401a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 0x7ff6401a0a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff63800eaa0 tx=0x7ff63800ef60 comp rx=0 tx=0).stop 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 shutdown_connections 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff640101c50 0x7ff6401a0a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff6280a2140 0x7ff6280a4600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff640101290 0x7ff6401999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 --2- 172.21.15.6:0/268361300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6401000a0 0x7ff640199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.555+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 >> 172.21.15.6:0/268361300 conn(0x7ff6400fb610 msgr2=0x7ff640108de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.556+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 shutdown_connections 2024-02-26T06:25:14.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.556+0000 7ff647400640 1 -- 172.21.15.6:0/268361300 wait complete. 2024-02-26T06:25:14.573 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 -- 172.21.15.6:0/3175116587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988101c00 msgr2=0x7f1988103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 --2- 172.21.15.6:0/3175116587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988101c00 0x7f1988103ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ca0 tx=0x7f197802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 -- 172.21.15.6:0/3175116587 shutdown_connections 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 --2- 172.21.15.6:0/3175116587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1988106e60 0x7f19881092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 --2- 172.21.15.6:0/3175116587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988104530 0x7f1988106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 --2- 172.21.15.6:0/3175116587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988101c00 0x7f1988103ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 -- 172.21.15.6:0/3175116587 >> 172.21.15.6:0/3175116587 conn(0x7f19880fb5f0 msgr2=0x7f19880fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:14.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 -- 172.21.15.6:0/3175116587 shutdown_connections 2024-02-26T06:25:14.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.770+0000 7f198ec00640 1 -- 172.21.15.6:0/3175116587 wait complete. 2024-02-26T06:25:14.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.771+0000 7f198ec00640 1 Processor -- start 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.771+0000 7f198ec00640 1 -- start start 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198ec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 0x7f1988199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198ec00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1988104530 0x7f1988199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988106e60 0x7f19881a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198ec00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881a0ef0 con 0x7f1988104530 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198ec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881a1060 con 0x7f1988101c00 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881a1220 con 0x7f1988106e60 2024-02-26T06:25:14.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f1987e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 0x7f1988199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f1987e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 0x7f1988199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:35022/0 (socket says 172.21.15.6:35022) 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f1987e00640 1 -- 172.21.15.6:0/939936404 learned_addr learned my addr 172.21.15.6:0/939936404 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f197f400640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1988104530 0x7f1988199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.772+0000 7f198cc00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988106e60 0x7f19881a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1987e00640 1 -- 172.21.15.6:0/939936404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988106e60 msgr2=0x7f19881a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1987e00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988106e60 0x7f19881a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1987e00640 1 -- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1988104530 msgr2=0x7f1988199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1987e00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1988104530 0x7f1988199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1987e00640 1 -- 172.21.15.6:0/939936404 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978009900 con 0x7f1988101c00 2024-02-26T06:25:14.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1987e00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 0x7f1988199400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1978032710 tx=0x7f1978034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:14.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780329b0 con 0x7f1988101c00 2024-02-26T06:25:14.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19881a14a0 con 0x7f1988101c00 2024-02-26T06:25:14.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1978003e50 con 0x7f1988101c00 2024-02-26T06:25:14.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.773+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978040450 con 0x7f1988101c00 2024-02-26T06:25:14.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.774+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881a1920 con 0x7f1988101c00 2024-02-26T06:25:14.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.775+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19881089b0 con 0x7f1988101c00 2024-02-26T06:25:14.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.777+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f197800a040 con 0x7f1988101c00 2024-02-26T06:25:14.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.779+0000 7f1985600640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f196c0a20c0 0x7f196c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:14.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.780+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19780f0300 con 0x7f1988101c00 2024-02-26T06:25:14.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.780+0000 7f197f400640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f196c0a20c0 0x7f196c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:14.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.780+0000 7f197f400640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f196c0a20c0 0x7f196c0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f198819a910 tx=0x7f1970005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:14.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:14.789+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19780b96d0 con 0x7f1988101c00 2024-02-26T06:25:15.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.004+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f198802cdd0 con 0x7f1988101c00 2024-02-26T06:25:15.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.012+0000 7f1985600640 1 -- 172.21.15.6:0/939936404 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f19780bcd90 con 0x7f1988101c00 2024-02-26T06:25:15.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:15.015 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2024-02-26T06:25:15.015 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:25:15.016 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2024-02-26T06:25:15.016 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2024-02-26T06:25:15.016 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:25:15.016 INFO:teuthology.orchestra.run.smithi006.stdout:] 2024-02-26T06:25:15.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f196c0a20c0 msgr2=0x7f196c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:15.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f196c0a20c0 0x7f196c0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f198819a910 tx=0x7f1970005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:15.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 msgr2=0x7f1988199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:15.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 0x7f1988199400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1978032710 tx=0x7f1978034a90 comp rx=0 tx=0).stop 2024-02-26T06:25:15.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 shutdown_connections 2024-02-26T06:25:15.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1988106e60 0x7f19881a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1988104530 0x7f1988199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f196c0a20c0 0x7f196c0a4580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 --2- 172.21.15.6:0/939936404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1988101c00 0x7f1988199400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.014+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 >> 172.21.15.6:0/939936404 conn(0x7f19880fb5f0 msgr2=0x7f19881012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:15.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.015+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 shutdown_connections 2024-02-26T06:25:15.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.015+0000 7f198ec00640 1 -- 172.21.15.6:0/939936404 wait complete. 2024-02-26T06:25:15.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T06:25:15.033 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T06:25:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1357461280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:14 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/268361300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:25:15.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 -- 172.21.15.6:0/3371058829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 msgr2=0x7f5c14100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:15.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 --2- 172.21.15.6:0/3371058829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 0x7f5c14100680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009900 tx=0x7f5c0402bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:15.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 -- 172.21.15.6:0/3371058829 shutdown_connections 2024-02-26T06:25:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 --2- 172.21.15.6:0/3371058829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c14100bc0 0x7f5c14101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 --2- 172.21.15.6:0/3371058829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 0x7f5c14100680 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 --2- 172.21.15.6:0/3371058829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c140ff8c0 0x7f5c140ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 -- 172.21.15.6:0/3371058829 >> 172.21.15.6:0/3371058829 conn(0x7f5c140fb460 msgr2=0x7f5c140fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 -- 172.21.15.6:0/3371058829 shutdown_connections 2024-02-26T06:25:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.247+0000 7f5c19200640 1 -- 172.21.15.6:0/3371058829 wait complete. 2024-02-26T06:25:15.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.248+0000 7f5c19200640 1 Processor -- start 2024-02-26T06:25:15.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.248+0000 7f5c19200640 1 -- start start 2024-02-26T06:25:15.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c19200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 0x7f5c14199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:15.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c19200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 0x7f5c141997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:15.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c19200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c14100bc0 0x7f5c141a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:15.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c19200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141a0d80 con 0x7f5c140ff8c0 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c19200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141a0ef0 con 0x7f5c14100220 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c19200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141a10b0 con 0x7f5c14100bc0 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 0x7f5c14199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c14100bc0 0x7f5c141a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 0x7f5c14199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52212/0 (socket says 172.21.15.6:52212) 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13400640 1 -- 172.21.15.6:0/2916584421 learned_addr learned my addr 172.21.15.6:0/2916584421 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c12a00640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 0x7f5c141997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13e00640 1 -- 172.21.15.6:0/2916584421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 msgr2=0x7f5c141997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13e00640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 0x7f5c141997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13e00640 1 -- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 msgr2=0x7f5c14199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13e00640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 0x7f5c14199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13e00640 1 -- 172.21.15.6:0/2916584421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009560 con 0x7f5c14100bc0 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.249+0000 7f5c13400640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 0x7f5c14199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:15.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.250+0000 7f5c13e00640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c14100bc0 0x7f5c141a0840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009fd0 tx=0x7f5c0c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:15.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.250+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00ce20 con 0x7f5c14100bc0 2024-02-26T06:25:15.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.250+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c0c005020 con 0x7f5c14100bc0 2024-02-26T06:25:15.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.250+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c010690 con 0x7f5c14100bc0 2024-02-26T06:25:15.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.250+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c141a1390 con 0x7f5c14100bc0 2024-02-26T06:25:15.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.250+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c141a1820 con 0x7f5c14100bc0 2024-02-26T06:25:15.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.251+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1410a610 con 0x7f5c14100bc0 2024-02-26T06:25:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.253+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5c0c005500 con 0x7f5c14100bc0 2024-02-26T06:25:15.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.254+0000 7f5c0be00640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5bf80a20c0 0x7f5bf80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:15.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.254+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c0c014070 con 0x7f5c14100bc0 2024-02-26T06:25:15.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.255+0000 7f5c13400640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5bf80a20c0 0x7f5bf80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:15.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.255+0000 7f5c13400640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5bf80a20c0 0x7f5bf80a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc011500 tx=0x7f5bfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.258+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c0c0973e0 con 0x7f5c14100bc0 2024-02-26T06:25:15.284 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1357461280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:15.285 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:14 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/268361300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:25:15.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1357461280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:15.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:15.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:25:15.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:14 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/268361300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:25:15.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:15.468+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5c1402ce40 con 0x7f5c14100bc0 2024-02-26T06:25:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.080+0000 7f5c0be00640 1 -- 172.21.15.6:0/2916584421 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5c0c005190 con 0x7f5c14100bc0 2024-02-26T06:25:16.081 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2024-02-26T06:25:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5bf80a20c0 msgr2=0x7f5bf80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5bf80a20c0 0x7f5bf80a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc011500 tx=0x7f5bfc005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c14100bc0 msgr2=0x7f5c141a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c14100bc0 0x7f5c141a0840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009fd0 tx=0x7f5c0c00ef10 comp rx=0 tx=0).stop 2024-02-26T06:25:16.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 shutdown_connections 2024-02-26T06:25:16.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c14100bc0 0x7f5c141a0840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5bf80a20c0 0x7f5bf80a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c14100220 0x7f5c141997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 --2- 172.21.15.6:0/2916584421 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140ff8c0 0x7f5c14199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.088+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 >> 172.21.15.6:0/2916584421 conn(0x7f5c140fb460 msgr2=0x7f5c140fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:16.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.089+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 shutdown_connections 2024-02-26T06:25:16.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.089+0000 7f5c19200640 1 -- 172.21.15.6:0/2916584421 wait complete. 2024-02-26T06:25:16.111 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T06:25:16.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 -- 172.21.15.6:0/1169814358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401033b0 msgr2=0x7fc1401037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 --2- 172.21.15.6:0/1169814358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401033b0 0x7fc1401037d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc130009900 tx=0x7fc13002bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 -- 172.21.15.6:0/1169814358 shutdown_connections 2024-02-26T06:25:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 --2- 172.21.15.6:0/1169814358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 0x7fc14010b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 --2- 172.21.15.6:0/1169814358 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401045a0 0x7fc140104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 --2- 172.21.15.6:0/1169814358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401033b0 0x7fc1401037d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 -- 172.21.15.6:0/1169814358 >> 172.21.15.6:0/1169814358 conn(0x7fc1400fe940 msgr2=0x7fc140100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:16.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 -- 172.21.15.6:0/1169814358 shutdown_connections 2024-02-26T06:25:16.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.321+0000 7fc146a00640 1 -- 172.21.15.6:0/1169814358 wait complete. 2024-02-26T06:25:16.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.322+0000 7fc146a00640 1 Processor -- start 2024-02-26T06:25:16.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.322+0000 7fc146a00640 1 -- start start 2024-02-26T06:25:16.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc146a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401033b0 0x7fc140199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:16.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc146a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401045a0 0x7fc1401999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:16.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc146a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 0x7fc1401a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc146a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401a0f60 con 0x7fc1401033b0 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc146a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401a10d0 con 0x7fc140104f60 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc146a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401a1290 con 0x7fc1401045a0 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401033b0 0x7fc140199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401033b0 0x7fc140199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52222/0 (socket says 172.21.15.6:52222) 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc13fe00640 1 -- 172.21.15.6:0/658185354 learned_addr learned my addr 172.21.15.6:0/658185354 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc144a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 0x7fc1401a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc13f400640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401045a0 0x7fc1401999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc144a00640 1 -- 172.21.15.6:0/658185354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401045a0 msgr2=0x7fc1401999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc144a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401045a0 0x7fc1401999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc144a00640 1 -- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401033b0 msgr2=0x7fc140199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc144a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401033b0 0x7fc140199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc144a00640 1 -- 172.21.15.6:0/658185354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130009560 con 0x7fc140104f60 2024-02-26T06:25:16.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.323+0000 7fc13f400640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401045a0 0x7fc1401999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.324+0000 7fc144a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 0x7fc1401a0a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc138009fd0 tx=0x7fc13800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.324+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800ce20 con 0x7fc140104f60 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.324+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc138005020 con 0x7fc140104f60 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.324+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1401a1570 con 0x7fc140104f60 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.324+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138010690 con 0x7fc140104f60 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.324+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1401a1a80 con 0x7fc140104f60 2024-02-26T06:25:16.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.325+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14010a6d0 con 0x7fc140104f60 2024-02-26T06:25:16.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.327+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc138004b20 con 0x7fc140104f60 2024-02-26T06:25:16.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.328+0000 7fc13cc00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1240a20c0 0x7fc1240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:16.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.328+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fc138014070 con 0x7fc140104f60 2024-02-26T06:25:16.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.329+0000 7fc13fe00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1240a20c0 0x7fc1240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:16.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.329+0000 7fc13fe00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1240a20c0 0x7fc1240a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc130006010 tx=0x7fc13002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:16.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.332+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1380973a0 con 0x7fc140104f60 2024-02-26T06:25:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:16 smithi090 ceph-mon[35681]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:16 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/939936404' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T06:25:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:16 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2916584421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:25:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:16 smithi006 ceph-mon[31790]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:16 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/939936404' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T06:25:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:16 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2916584421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:25:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:16 smithi167 ceph-mon[36229]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:16 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/939936404' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T06:25:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:16 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2916584421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:25:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:16.569+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc14002cdd0 con 0x7fc140104f60 2024-02-26T06:25:17.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.186+0000 7fc13cc00640 1 -- 172.21.15.6:0/658185354 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc13809aa60 con 0x7fc140104f60 2024-02-26T06:25:17.186 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2024-02-26T06:25:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1240a20c0 msgr2=0x7fc1240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1240a20c0 0x7fc1240a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc130006010 tx=0x7fc13002c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 msgr2=0x7fc1401a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 0x7fc1401a0a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc138009fd0 tx=0x7fc13800ef10 comp rx=0 tx=0).stop 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 shutdown_connections 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1240a20c0 0x7fc1240a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc140104f60 0x7fc1401a0a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1401045a0 0x7fc1401999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 --2- 172.21.15.6:0/658185354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1401033b0 0x7fc140199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.194+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 >> 172.21.15.6:0/658185354 conn(0x7fc1400fe940 msgr2=0x7fc1401078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.195+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 shutdown_connections 2024-02-26T06:25:17.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.195+0000 7fc146a00640 1 -- 172.21.15.6:0/658185354 wait complete. 2024-02-26T06:25:17.225 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T06:25:17.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.405+0000 7fd88cc00640 1 -- 172.21.15.6:0/905154982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 msgr2=0x7fd888104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:17.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.405+0000 7fd88cc00640 1 --2- 172.21.15.6:0/905154982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd888104850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:17.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 -- 172.21.15.6:0/905154982 shutdown_connections 2024-02-26T06:25:17.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 --2- 172.21.15.6:0/905154982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd888104d90 0x7fd88810b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 --2- 172.21.15.6:0/905154982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd888104850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 --2- 172.21.15.6:0/905154982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8881031e0 0x7fd888103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 -- 172.21.15.6:0/905154982 >> 172.21.15.6:0/905154982 conn(0x7fd8880fe750 msgr2=0x7fd888100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 -- 172.21.15.6:0/905154982 shutdown_connections 2024-02-26T06:25:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.406+0000 7fd88cc00640 1 -- 172.21.15.6:0/905154982 wait complete. 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 Processor -- start 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 -- start start 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8881031e0 0x7fd888199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd8881997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd888104d90 0x7fd8881a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881a0d50 con 0x7fd8881043d0 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881a0ec0 con 0x7fd888104d90 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd88cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881a1080 con 0x7fd8881031e0 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd87fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd8881997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd87fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd8881997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38914/0 (socket says 172.21.15.6:38914) 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd87fe00640 1 -- 172.21.15.6:0/2645860905 learned_addr learned my addr 172.21.15.6:0/2645860905 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd887400640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8881031e0 0x7fd888199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.407+0000 7fd887e00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd888104d90 0x7fd8881a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd87fe00640 1 -- 172.21.15.6:0/2645860905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8881031e0 msgr2=0x7fd888199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd87fe00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8881031e0 0x7fd888199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd87fe00640 1 -- 172.21.15.6:0/2645860905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd888104d90 msgr2=0x7fd8881a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd87fe00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd888104d90 0x7fd8881a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd87fe00640 1 -- 172.21.15.6:0/2645860905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878009490 con 0x7fd8881043d0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd87fe00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd8881997a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd87002c6a0 tx=0x7fd8700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700044e0 con 0x7fd8881043d0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd870004ac0 con 0x7fd8881043d0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.408+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87003e450 con 0x7fd8881043d0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.409+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd870009560 con 0x7fd8881043d0 2024-02-26T06:25:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.409+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8881a1630 con 0x7fd8881043d0 2024-02-26T06:25:17.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.410+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88810a500 con 0x7fd8881043d0 2024-02-26T06:25:17.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.412+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd870003680 con 0x7fd8881043d0 2024-02-26T06:25:17.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.413+0000 7fd884c00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd8680a20c0 0x7fd8680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:17.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.413+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fd8700f0830 con 0x7fd8881043d0 2024-02-26T06:25:17.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.413+0000 7fd887400640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd8680a20c0 0x7fd8680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:17.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.414+0000 7fd887400640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd8680a20c0 0x7fd8680a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd888104230 tx=0x7fd878005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:17.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.416+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8700b9910 con 0x7fd8881043d0 2024-02-26T06:25:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:25:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T06:25:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T06:25:17.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:17.655+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd88802ceb0 con 0x7fd8881043d0 2024-02-26T06:25:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T06:25:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T06:25:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T06:25:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:17.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.231+0000 7fd884c00640 1 -- 172.21.15.6:0/2645860905 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd8700430c0 con 0x7fd8881043d0 2024-02-26T06:25:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 2 and data pool 3 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd8680a20c0 msgr2=0x7fd8680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd8680a20c0 0x7fd8680a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd888104230 tx=0x7fd878005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 msgr2=0x7fd8881997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd8881997a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd87002c6a0 tx=0x7fd8700042f0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 shutdown_connections 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd8680a20c0 0x7fd8680a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd888104d90 0x7fd8881a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8881043d0 0x7fd8881997a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 --2- 172.21.15.6:0/2645860905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8881031e0 0x7fd888199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 >> 172.21.15.6:0/2645860905 conn(0x7fd8880fe750 msgr2=0x7fd8881076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 shutdown_connections 2024-02-26T06:25:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.235+0000 7fd88cc00640 1 -- 172.21.15.6:0/2645860905 wait complete. 2024-02-26T06:25:18.258 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:25:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 -- 172.21.15.6:0/729099224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98101080 msgr2=0x7fcc981014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 --2- 172.21.15.6:0/729099224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98101080 0x7fcc981014a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009900 tx=0x7fcc8c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 -- 172.21.15.6:0/729099224 shutdown_connections 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 --2- 172.21.15.6:0/729099224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102c30 0x7fcc981094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 --2- 172.21.15.6:0/729099224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102270 0x7fcc981026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 --2- 172.21.15.6:0/729099224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98101080 0x7fcc981014a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 -- 172.21.15.6:0/729099224 >> 172.21.15.6:0/729099224 conn(0x7fcc980fc610 msgr2=0x7fcc980fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 -- 172.21.15.6:0/729099224 shutdown_connections 2024-02-26T06:25:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.447+0000 7fcc9f000640 1 -- 172.21.15.6:0/729099224 wait complete. 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 Processor -- start 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 -- start start 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98101080 0x7fcc981993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 0x7fcc981998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98102c30 0x7fcc981a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a0ea0 con 0x7fcc98102c30 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a1010 con 0x7fcc98102270 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.448+0000 7fcc9f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a11d0 con 0x7fcc98101080 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 0x7fcc981998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9e600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98102c30 0x7fcc981a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 0x7fcc981998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34488/0 (socket says 172.21.15.6:34488) 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 -- 172.21.15.6:0/3187282762 learned_addr learned my addr 172.21.15.6:0/3187282762 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9dc00640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98101080 0x7fcc981993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 -- 172.21.15.6:0/3187282762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98101080 msgr2=0x7fcc981993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98101080 0x7fcc981993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 -- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98102c30 msgr2=0x7fcc981a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98102c30 0x7fcc981a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 -- 172.21.15.6:0/3187282762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c009560 con 0x7fcc98102270 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9e600640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98102c30 0x7fcc981a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.449+0000 7fcc9d200640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 0x7fcc981998f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400e9f0 tx=0x7fcc8400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.450+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84009720 con 0x7fcc98102270 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.450+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc8400cd80 con 0x7fcc98102270 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.450+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84010690 con 0x7fcc98102270 2024-02-26T06:25:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.450+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc981a14b0 con 0x7fcc98102270 2024-02-26T06:25:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.450+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc981a19c0 con 0x7fcc98102270 2024-02-26T06:25:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.451+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9810cae0 con 0x7fcc98102270 2024-02-26T06:25:18.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.453+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fcc840046c0 con 0x7fcc98102270 2024-02-26T06:25:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.454+0000 7fcc96000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcc800a2130 0x7fcc800a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.455+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc84014070 con 0x7fcc98102270 2024-02-26T06:25:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.455+0000 7fcc9dc00640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcc800a2130 0x7fcc800a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.455+0000 7fcc9dc00640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcc800a2130 0x7fcc800a45f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02c240 tx=0x7fcc8c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:18.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.459+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc84096f70 con 0x7fcc98102270 2024-02-26T06:25:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: pgmap v314: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/658185354' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2645860905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:25:18.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:18 smithi167 ceph-mon[36229]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T06:25:18.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: pgmap v314: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:18.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/658185354' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:25:18.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:25:18.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2645860905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:25:18.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:18 smithi090 ceph-mon[35681]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T06:25:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.660+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc9810cc70 con 0x7fcc98102270 2024-02-26T06:25:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.662+0000 7fcc96000640 1 -- 172.21.15.6:0/3187282762 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22127 (secure 0 0 0) 0x7fcc8409a630 con 0x7fcc98102270 2024-02-26T06:25:18.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:18.662 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":84,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:25:17.656543+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcc800a2130 msgr2=0x7fcc800a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcc800a2130 0x7fcc800a45f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02c240 tx=0x7fcc8c02c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 msgr2=0x7fcc981998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 0x7fcc981998f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400e9f0 tx=0x7fcc8400edf0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 shutdown_connections 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc98102c30 0x7fcc981a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcc800a2130 0x7fcc800a45f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc98102270 0x7fcc981998f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 --2- 172.21.15.6:0/3187282762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98101080 0x7fcc981993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.665+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 >> 172.21.15.6:0/3187282762 conn(0x7fcc980fc610 msgr2=0x7fcc980733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.666+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 shutdown_connections 2024-02-26T06:25:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.666+0000 7fcc9f000640 1 -- 172.21.15.6:0/3187282762 wait complete. 2024-02-26T06:25:18.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: pgmap v314: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:18.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/658185354' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:25:18.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T06:25:18.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T06:25:18.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2645860905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T06:25:18.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:18 smithi006 ceph-mon[31790]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T06:25:18.686 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:25:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.912+0000 7f75ebe00640 1 -- 172.21.15.6:0/3920582519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 msgr2=0x7f75ec106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.912+0000 7f75ebe00640 1 --2- 172.21.15.6:0/3920582519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 0x7f75ec106830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009900 tx=0x7f75d402bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 -- 172.21.15.6:0/3920582519 shutdown_connections 2024-02-26T06:25:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 --2- 172.21.15.6:0/3920582519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 0x7f75ec1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 --2- 172.21.15.6:0/3920582519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 0x7f75ec106830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 --2- 172.21.15.6:0/3920582519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 -- 172.21.15.6:0/3920582519 >> 172.21.15.6:0/3920582519 conn(0x7f75ec0fb500 msgr2=0x7f75ec0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 -- 172.21.15.6:0/3920582519 shutdown_connections 2024-02-26T06:25:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.913+0000 7f75ebe00640 1 -- 172.21.15.6:0/3920582519 wait complete. 2024-02-26T06:25:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.914+0000 7f75ebe00640 1 Processor -- start 2024-02-26T06:25:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.914+0000 7f75ebe00640 1 -- start start 2024-02-26T06:25:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 0x7f75ec1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 0x7f75ec1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ebe00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a0e90 con 0x7f75ec106d70 2024-02-26T06:25:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ebe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a1000 con 0x7f75ec101b10 2024-02-26T06:25:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ebe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a11c0 con 0x7f75ec104440 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75eaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75eaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34508/0 (socket says 172.21.15.6:34508) 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75eaa00640 1 -- 172.21.15.6:0/2993476497 learned_addr learned my addr 172.21.15.6:0/2993476497 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75eb400640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 0x7f75ec1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75ea000640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 0x7f75ec1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75eaa00640 1 -- 172.21.15.6:0/2993476497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 msgr2=0x7f75ec1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.915+0000 7f75eaa00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 0x7f75ec1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75eaa00640 1 -- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 msgr2=0x7f75ec1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75eaa00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 0x7f75ec1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75eaa00640 1 -- 172.21.15.6:0/2993476497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009560 con 0x7f75ec101b10 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75eb400640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 0x7f75ec1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75eaa00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec1993a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00e9f0 tx=0x7f75dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc009720 con 0x7f75ec101b10 2024-02-26T06:25:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75dc00cd80 con 0x7f75ec101b10 2024-02-26T06:25:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc010690 con 0x7f75ec101b10 2024-02-26T06:25:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec1a14a0 con 0x7f75ec101b10 2024-02-26T06:25:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.916+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1a1980 con 0x7f75ec101b10 2024-02-26T06:25:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.918+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec1088c0 con 0x7f75ec101b10 2024-02-26T06:25:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.919+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f75dc0046c0 con 0x7f75ec101b10 2024-02-26T06:25:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.921+0000 7f75e3400640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f75d00a2130 0x7f75d00a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.921+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f75dc014070 con 0x7f75ec101b10 2024-02-26T06:25:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.921+0000 7f75ea000640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f75d00a2130 0x7f75d00a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.922+0000 7f75ea000640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f75d00a2130 0x7f75d00a45f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005ab0 tx=0x7f75d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:18.925+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75dc096f70 con 0x7f75ec101b10 2024-02-26T06:25:19.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.127+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75ec10ca90 con 0x7f75ec101b10 2024-02-26T06:25:19.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.129+0000 7f75e3400640 1 -- 172.21.15.6:0/2993476497 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22127 (secure 0 0 0) 0x7f75dc09a630 con 0x7f75ec101b10 2024-02-26T06:25:19.129 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:19.129 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":84,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:25:17.656543+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:25:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f75d00a2130 msgr2=0x7f75d00a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f75d00a2130 0x7f75d00a45f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005ab0 tx=0x7f75d4005fd0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 msgr2=0x7f75ec1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec1993a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00e9f0 tx=0x7f75dc00edf0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 shutdown_connections 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec106d70 0x7f75ec1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec104440 0x7f75ec1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f75d00a2130 0x7f75d00a45f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 --2- 172.21.15.6:0/2993476497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec101b10 0x7f75ec1993a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 >> 172.21.15.6:0/2993476497 conn(0x7f75ec0fb500 msgr2=0x7f75ec101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 shutdown_connections 2024-02-26T06:25:19.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.132+0000 7f75ebe00640 1 -- 172.21.15.6:0/2993476497 wait complete. 2024-02-26T06:25:19.150 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 -- 172.21.15.6:0/2343626868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4498104d70 msgr2=0x7f449810b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 --2- 172.21.15.6:0/2343626868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4498104d70 0x7f449810b600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4490009be0 tx=0x7f449002bef0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 -- 172.21.15.6:0/2343626868 shutdown_connections 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 --2- 172.21.15.6:0/2343626868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4498104d70 0x7f449810b600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 --2- 172.21.15.6:0/2343626868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44981043b0 0x7f4498104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 --2- 172.21.15.6:0/2343626868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981031c0 0x7f44981035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 -- 172.21.15.6:0/2343626868 >> 172.21.15.6:0/2343626868 conn(0x7f44980fe750 msgr2=0x7f4498100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 -- 172.21.15.6:0/2343626868 shutdown_connections 2024-02-26T06:25:19.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.372+0000 7f449e800640 1 -- 172.21.15.6:0/2343626868 wait complete. 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.373+0000 7f449e800640 1 Processor -- start 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.373+0000 7f449e800640 1 -- start start 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981031c0 0x7f4498199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44981043b0 0x7f4498199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449e800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 0x7f44981a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449e800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44981a0cf0 con 0x7f4498104d70 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44981a0e60 con 0x7f44981043b0 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44981a1020 con 0x7f44981031c0 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 0x7f44981a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 0x7f44981a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38966/0 (socket says 172.21.15.6:38966) 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449de00640 1 -- 172.21.15.6:0/558182355 learned_addr learned my addr 172.21.15.6:0/558182355 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449ca00640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44981043b0 0x7f4498199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.374+0000 7f449d400640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981031c0 0x7f4498199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449ca00640 1 -- 172.21.15.6:0/558182355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981031c0 msgr2=0x7f4498199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449ca00640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981031c0 0x7f4498199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449ca00640 1 -- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 msgr2=0x7f44981a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449ca00640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 0x7f44981a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449ca00640 1 -- 172.21.15.6:0/558182355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4490009840 con 0x7f44981043b0 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449de00640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 0x7f44981a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:19.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449ca00640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44981043b0 0x7f4498199740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f448400e790 tx=0x7f448400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400ccc0 con 0x7f44981043b0 2024-02-26T06:25:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f448400eea0 con 0x7f44981043b0 2024-02-26T06:25:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44981a1300 con 0x7f44981043b0 2024-02-26T06:25:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484018730 con 0x7f44981043b0 2024-02-26T06:25:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.375+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44981a1860 con 0x7f44981043b0 2024-02-26T06:25:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.376+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449810a4e0 con 0x7f44981043b0 2024-02-26T06:25:19.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.379+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4484010430 con 0x7f44981043b0 2024-02-26T06:25:19.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.380+0000 7f4496000640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f44800a2140 0x7f44800a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:19.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.381+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4484014070 con 0x7f44981043b0 2024-02-26T06:25:19.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.381+0000 7f449d400640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f44800a2140 0x7f44800a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:19.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.381+0000 7f449d400640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f44800a2140 0x7f44800a4600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4498104210 tx=0x7f448c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:19.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.384+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4484052d20 con 0x7f44981043b0 2024-02-26T06:25:19.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: pgmap v316: 97 pgs: 17 creating+peering, 3 active+clean, 77 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:19.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T06:25:19.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2645860905' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T06:25:19.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T06:25:19.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T06:25:19.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3187282762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:19.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:19 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2993476497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:19.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: pgmap v316: 97 pgs: 17 creating+peering, 3 active+clean, 77 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:19.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T06:25:19.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2645860905' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T06:25:19.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T06:25:19.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T06:25:19.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3187282762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:19.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:19 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2993476497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: pgmap v316: 97 pgs: 17 creating+peering, 3 active+clean, 77 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:25:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T06:25:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2645860905' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T06:25:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T06:25:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T06:25:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3187282762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:19.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:19 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2993476497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:19.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:19.627+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f449802ce40 con 0x7f44981043b0 2024-02-26T06:25:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.265+0000 7f4496000640 1 -- 172.21.15.6:0/558182355 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4484096e30 con 0x7f44981043b0 2024-02-26T06:25:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f44800a2140 msgr2=0x7f44800a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f44800a2140 0x7f44800a4600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4498104210 tx=0x7f448c005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44981043b0 msgr2=0x7f4498199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44981043b0 0x7f4498199740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f448400e790 tx=0x7f448400ec50 comp rx=0 tx=0).stop 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 shutdown_connections 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4498104d70 0x7f44981a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f44800a2140 0x7f44800a4600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44981043b0 0x7f4498199740 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 --2- 172.21.15.6:0/558182355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981031c0 0x7f4498199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 >> 172.21.15.6:0/558182355 conn(0x7f44980fe750 msgr2=0x7f44981076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 shutdown_connections 2024-02-26T06:25:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.269+0000 7f449e800640 1 -- 172.21.15.6:0/558182355 wait complete. 2024-02-26T06:25:20.289 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-02-26T06:25:20.289 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-02-26T06:25:20.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:20 smithi167 ceph-mon[36229]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T06:25:20.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:20 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] up:active 2024-02-26T06:25:20.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:20 smithi167 ceph-mon[36229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:20.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:20 smithi167 ceph-mon[36229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:20.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:20 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/558182355' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:25:20.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:20 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:25:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 -- 172.21.15.6:0/4036697738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 msgr2=0x7f9be41069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 --2- 172.21.15.6:0/4036697738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be41069a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009900 tx=0x7f9bd402bc40 comp rx=0 tx=0).stop 2024-02-26T06:25:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 -- 172.21.15.6:0/4036697738 shutdown_connections 2024-02-26T06:25:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 --2- 172.21.15.6:0/4036697738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be4106ee0 0x7f9be41092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 --2- 172.21.15.6:0/4036697738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be41069a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 --2- 172.21.15.6:0/4036697738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be40fedd0 0x7f9be40ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.517+0000 7f9be9200640 1 -- 172.21.15.6:0/4036697738 >> 172.21.15.6:0/4036697738 conn(0x7f9be40fa930 msgr2=0x7f9be40fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:20.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.518+0000 7f9be9200640 1 -- 172.21.15.6:0/4036697738 shutdown_connections 2024-02-26T06:25:20.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.518+0000 7f9be9200640 1 -- 172.21.15.6:0/4036697738 wait complete. 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.519+0000 7f9be9200640 1 Processor -- start 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.519+0000 7f9be9200640 1 -- start start 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.519+0000 7f9be9200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 0x7f9be41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.519+0000 7f9be9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.519+0000 7f9be9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be4106ee0 0x7f9be41a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be9200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a0fb0 con 0x7f9be40fedd0 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be9200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a1120 con 0x7f9be4106ee0 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be9200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a12e0 con 0x7f9be40ff7a0 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be1e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 0x7f9be41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:20.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be1e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 0x7f9be41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38978/0 (socket says 172.21.15.6:38978) 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be1e00640 1 -- 172.21.15.6:0/3513744029 learned_addr learned my addr 172.21.15.6:0/3513744029 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be2800640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be4106ee0 0x7f9be41a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be1400640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be2800640 1 -- 172.21.15.6:0/3513744029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 msgr2=0x7f9be4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be2800640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be2800640 1 -- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 msgr2=0x7f9be41994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be2800640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 0x7f9be41994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be2800640 1 -- 172.21.15.6:0/3513744029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4009560 con 0x7f9be4106ee0 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be1400640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.520+0000 7f9be1e00640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 0x7f9be41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.521+0000 7f9be2800640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be4106ee0 0x7f9be41a0a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00ea80 tx=0x7f9bdc00ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:20.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.521+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00cea0 con 0x7f9be4106ee0 2024-02-26T06:25:20.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.521+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bdc005020 con 0x7f9be4106ee0 2024-02-26T06:25:20.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.521+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc010690 con 0x7f9be4106ee0 2024-02-26T06:25:20.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.521+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be41a15c0 con 0x7f9be4106ee0 2024-02-26T06:25:20.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.521+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be41a1a50 con 0x7f9be4106ee0 2024-02-26T06:25:20.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.522+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be410cbc0 con 0x7f9be4106ee0 2024-02-26T06:25:20.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.524+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9bdc005500 con 0x7f9be4106ee0 2024-02-26T06:25:20.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.526+0000 7f9bdaa00640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9bc80a2140 0x7f9bc80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:20.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.526+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9bdc014070 con 0x7f9be4106ee0 2024-02-26T06:25:20.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.526+0000 7f9be1e00640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9bc80a2140 0x7f9bc80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:20.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.527+0000 7f9be1e00640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9bc80a2140 0x7f9bc80a4600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc011550 tx=0x7f9bcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:20.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.529+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bdc097370 con 0x7f9be4106ee0 2024-02-26T06:25:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:20 smithi090 ceph-mon[35681]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T06:25:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:20 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] up:active 2024-02-26T06:25:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:20 smithi090 ceph-mon[35681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:20 smithi090 ceph-mon[35681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:20 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/558182355' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:25:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:20 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:25:20.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:20 smithi006 ceph-mon[31790]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T06:25:20.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:20 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] up:active 2024-02-26T06:25:20.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:20 smithi006 ceph-mon[31790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:20.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:20 smithi006 ceph-mon[31790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:20.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:20 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/558182355' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:25:20.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:20 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T06:25:20.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:20.770+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f9be41a1cd0 con 0x7f9be4106ee0 2024-02-26T06:25:21.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.269+0000 7f9bdaa00640 1 -- 172.21.15.6:0/3513744029 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f9bdc005190 con 0x7f9be4106ee0 2024-02-26T06:25:21.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9bc80a2140 msgr2=0x7f9bc80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9bc80a2140 0x7f9bc80a4600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc011550 tx=0x7f9bcc005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be4106ee0 msgr2=0x7f9be41a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be4106ee0 0x7f9be41a0a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00ea80 tx=0x7f9bdc00ef40 comp rx=0 tx=0).stop 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 shutdown_connections 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9bc80a2140 0x7f9bc80a4600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9be4106ee0 0x7f9be41a0a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be40ff7a0 0x7f9be4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 --2- 172.21.15.6:0/3513744029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be40fedd0 0x7f9be41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 >> 172.21.15.6:0/3513744029 conn(0x7f9be40fa930 msgr2=0x7f9be40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.273+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 shutdown_connections 2024-02-26T06:25:21.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.274+0000 7f9be9200640 1 -- 172.21.15.6:0/3513744029 wait complete. 2024-02-26T06:25:21.298 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-26T06:25:21.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: pgmap v319: 97 pgs: 17 creating+peering, 40 active+clean, 40 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-02-26T06:25:21.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-26T06:25:21.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: Cluster is now healthy 2024-02-26T06:25:21.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T06:25:21.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:21.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3513744029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-26T06:25:21.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:21 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-26T06:25:21.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 -- 172.21.15.6:0/3040436762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 msgr2=0x7f5f041095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:21.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/3040436762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 0x7f5f041095c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009ca0 tx=0x7f5efc02c100 comp rx=0 tx=0).stop 2024-02-26T06:25:21.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 -- 172.21.15.6:0/3040436762 shutdown_connections 2024-02-26T06:25:21.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/3040436762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 0x7f5f041095c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/3040436762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f04102370 0x7f5f041027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/3040436762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f041015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 -- 172.21.15.6:0/3040436762 >> 172.21.15.6:0/3040436762 conn(0x7f5f040fc6d0 msgr2=0x7f5f040feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:21.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 -- 172.21.15.6:0/3040436762 shutdown_connections 2024-02-26T06:25:21.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.499+0000 7f5f0ae00640 1 -- 172.21.15.6:0/3040436762 wait complete. 2024-02-26T06:25:21.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.500+0000 7f5f0ae00640 1 Processor -- start 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.500+0000 7f5f0ae00640 1 -- start start 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.500+0000 7f5f0ae00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f04199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.500+0000 7f5f0ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f04102370 0x7f5f04199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f0ae00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 0x7f5f041a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f0ae00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f041a0eb0 con 0x7f5f04102d30 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f0ae00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f041a1020 con 0x7f5f04101180 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f0ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f041a11e0 con 0x7f5f04102370 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f04199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f04199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34564/0 (socket says 172.21.15.6:34564) 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 -- 172.21.15.6:0/1258766979 learned_addr learned my addr 172.21.15.6:0/1258766979 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03400640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f04102370 0x7f5f04199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f08e00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 0x7f5f041a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 -- 172.21.15.6:0/1258766979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f04102370 msgr2=0x7f5f04199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:21.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f04102370 0x7f5f04199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 -- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 msgr2=0x7f5f041a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:21.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 0x7f5f041a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:21.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.501+0000 7f5f03e00640 1 -- 172.21.15.6:0/1258766979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc009900 con 0x7f5f04101180 2024-02-26T06:25:21.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.502+0000 7f5f03e00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f04199410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e9c0 tx=0x7f5ef400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:21.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.502+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4009700 con 0x7f5f04101180 2024-02-26T06:25:21.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.502+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f041a14c0 con 0x7f5f04101180 2024-02-26T06:25:21.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.502+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef400cd80 con 0x7f5f04101180 2024-02-26T06:25:21.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.502+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4010430 con 0x7f5f04101180 2024-02-26T06:25:21.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.502+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f041a1a20 con 0x7f5f04101180 2024-02-26T06:25:21.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.503+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0410cb90 con 0x7f5f04101180 2024-02-26T06:25:21.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.505+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5ef4003680 con 0x7f5f04101180 2024-02-26T06:25:21.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.507+0000 7f5f00c00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5ee80a2140 0x7f5ee80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:21.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.507+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ef4014070 con 0x7f5f04101180 2024-02-26T06:25:21.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.507+0000 7f5f03400640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5ee80a2140 0x7f5ee80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:21.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.508+0000 7f5f03400640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5ee80a2140 0x7f5ee80a4600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f0419a8d0 tx=0x7f5ef0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:21.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.511+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ef4096f60 con 0x7f5f04101180 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: pgmap v319: 97 pgs: 17 creating+peering, 40 active+clean, 40 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: Cluster is now healthy 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3513744029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-26T06:25:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:21 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: pgmap v319: 97 pgs: 17 creating+peering, 40 active+clean, 40 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: Cluster is now healthy 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3513744029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-26T06:25:21.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:21 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-26T06:25:21.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:21.754+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f5f0410cd70 con 0x7f5f04101180 2024-02-26T06:25:22.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.279+0000 7f5f00c00640 1 -- 172.21.15.6:0/1258766979 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5ef409a620 con 0x7f5f04101180 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5ee80a2140 msgr2=0x7f5ee80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5ee80a2140 0x7f5ee80a4600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f0419a8d0 tx=0x7f5ef0005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 msgr2=0x7f5f04199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f04199410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e9c0 tx=0x7f5ef400ee80 comp rx=0 tx=0).stop 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 shutdown_connections 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f04102d30 0x7f5f041a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f04102370 0x7f5f04199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5ee80a2140 0x7f5ee80a4600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 --2- 172.21.15.6:0/1258766979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f04101180 0x7f5f04199410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.283+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 >> 172.21.15.6:0/1258766979 conn(0x7f5f040fc6d0 msgr2=0x7f5f040733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:22.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.284+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 shutdown_connections 2024-02-26T06:25:22.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.284+0000 7f5f0ae00640 1 -- 172.21.15.6:0/1258766979 wait complete. 2024-02-26T06:25:22.307 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T06:25:22.307 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-26T06:25:22.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T06:25:22.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:22.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-02-26T06:25:22.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-02-26T06:25:22.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1258766979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:25:22.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:22 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:25:22.310 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-26T06:25:22.446 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T06:25:22.446 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-26T06:25:22.446 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T06:25:22.446 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:22.446 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-02-26T06:25:22.446 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-02-26T06:25:22.447 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1258766979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:25:22.447 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:25:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1258766979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:25:22.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:22 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T06:25:22.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 -- 172.21.15.6:0/688074323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 msgr2=0x7f1fd410b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:22.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 --2- 172.21.15.6:0/688074323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd410b7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009860 tx=0x7f1fcc02bb70 comp rx=0 tx=0).stop 2024-02-26T06:25:22.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 -- 172.21.15.6:0/688074323 shutdown_connections 2024-02-26T06:25:22.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 --2- 172.21.15.6:0/688074323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd410b7f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 --2- 172.21.15.6:0/688074323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 0x7f1fd4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 --2- 172.21.15.6:0/688074323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 0x7f1fd41037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 -- 172.21.15.6:0/688074323 >> 172.21.15.6:0/688074323 conn(0x7f1fd40fe940 msgr2=0x7f1fd4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:22.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 -- 172.21.15.6:0/688074323 shutdown_connections 2024-02-26T06:25:22.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.509+0000 7f1fd9400640 1 -- 172.21.15.6:0/688074323 wait complete. 2024-02-26T06:25:22.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 Processor -- start 2024-02-26T06:25:22.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 -- start start 2024-02-26T06:25:22.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 0x7f1fd4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:22.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 0x7f1fd4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:22.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd41a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:22.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd41a0ef0 con 0x7f1fd41045a0 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd41a1060 con 0x7f1fd4104f60 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.510+0000 7f1fd9400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd41a1220 con 0x7f1fd41033b0 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fc9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 0x7f1fd4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd41a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fc9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 0x7f1fd4199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39010/0 (socket says 172.21.15.6:39010) 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fc9e00640 1 -- 172.21.15.6:0/28633941 learned_addr learned my addr 172.21.15.6:0/28633941 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 0x7f1fd4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 -- 172.21.15.6:0/28633941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 msgr2=0x7f1fd41a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd41a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 -- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 msgr2=0x7f1fd4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 0x7f1fd4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 -- 172.21.15.6:0/28633941 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc009490 con 0x7f1fd41033b0 2024-02-26T06:25:22.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd3200640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd41a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:22.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.511+0000 7f1fd2800640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 0x7f1fd4199400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005fd0 tx=0x7f1fc400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:22.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.512+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400cea0 con 0x7f1fd41033b0 2024-02-26T06:25:22.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.512+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd41a1500 con 0x7f1fd41033b0 2024-02-26T06:25:22.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.512+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fc4003e50 con 0x7f1fd41033b0 2024-02-26T06:25:22.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.512+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4010720 con 0x7f1fd41033b0 2024-02-26T06:25:22.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.512+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd41a1a60 con 0x7f1fd41033b0 2024-02-26T06:25:22.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.513+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd410a6d0 con 0x7f1fd41033b0 2024-02-26T06:25:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.515+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1fc4004380 con 0x7f1fd41033b0 2024-02-26T06:25:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.517+0000 7f1fcbe00640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1fb80a20c0 0x7f1fb80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.517+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1fc4014070 con 0x7f1fd41033b0 2024-02-26T06:25:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.517+0000 7f1fc9e00640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1fb80a20c0 0x7f1fb80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.518+0000 7f1fc9e00640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1fb80a20c0 0x7f1fb80a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fd419a910 tx=0x7f1fbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.519+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fc4097500 con 0x7f1fd41033b0 2024-02-26T06:25:22.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:22.761+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1fd410ef40 con 0x7f1fd41033b0 2024-02-26T06:25:22.865 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: 2024-02-26T06:25:22.445+0000 7fa09026a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T06:25:22.865 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: 2024-02-26T06:25:22.445+0000 7fa09026a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-02-26T06:25:23.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.293+0000 7f1fcbe00640 1 -- 172.21.15.6:0/28633941 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1fc409abc0 con 0x7f1fd41033b0 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.297+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1fb80a20c0 msgr2=0x7f1fb80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.297+0000 7f1fd9400640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1fb80a20c0 0x7f1fb80a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fd419a910 tx=0x7f1fbc009260 comp rx=0 tx=0).stop 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.297+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 msgr2=0x7f1fd4199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.297+0000 7f1fd9400640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 0x7f1fd4199400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005fd0 tx=0x7f1fc400ef90 comp rx=0 tx=0).stop 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 shutdown_connections 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1fb80a20c0 0x7f1fb80a4580 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fd4104f60 0x7f1fd41a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fd41045a0 0x7f1fd4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 --2- 172.21.15.6:0/28633941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fd41033b0 0x7f1fd4199400 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 >> 172.21.15.6:0/28633941 conn(0x7f1fd40fe940 msgr2=0x7f1fd41078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 shutdown_connections 2024-02-26T06:25:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.298+0000 7f1fd9400640 1 -- 172.21.15.6:0/28633941 wait complete. 2024-02-26T06:25:23.324 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-02-26T06:25:23.325 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-26T06:25:23.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.533+0000 7f186a000640 1 -- 172.21.15.6:0/1424182025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1864104f60 msgr2=0x7f186410b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.533+0000 7f186a000640 1 --2- 172.21.15.6:0/1424182025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1864104f60 0x7f186410b7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f185c009ca0 tx=0x7f185c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 -- 172.21.15.6:0/1424182025 shutdown_connections 2024-02-26T06:25:23.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 --2- 172.21.15.6:0/1424182025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1864104f60 0x7f186410b7f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 --2- 172.21.15.6:0/1424182025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641045a0 0x7f1864104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 --2- 172.21.15.6:0/1424182025 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18641033b0 0x7f18641037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 -- 172.21.15.6:0/1424182025 >> 172.21.15.6:0/1424182025 conn(0x7f18640fe940 msgr2=0x7f1864100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:23.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 -- 172.21.15.6:0/1424182025 shutdown_connections 2024-02-26T06:25:23.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.534+0000 7f186a000640 1 -- 172.21.15.6:0/1424182025 wait complete. 2024-02-26T06:25:23.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 Processor -- start 2024-02-26T06:25:23.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 -- start start 2024-02-26T06:25:23.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18641033b0 0x7f1864199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:23.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641045a0 0x7f1864199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1864104f60 0x7f18641a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a0ef0 con 0x7f18641033b0 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a1060 con 0x7f1864104f60 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.535+0000 7f186a000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a1220 con 0x7f18641045a0 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18641033b0 0x7f1864199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1864104f60 0x7f18641a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18641033b0 0x7f1864199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39028/0 (socket says 172.21.15.6:39028) 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1862a00640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641045a0 0x7f1864199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863400640 1 -- 172.21.15.6:0/758682829 learned_addr learned my addr 172.21.15.6:0/758682829 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 -- 172.21.15.6:0/758682829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641045a0 msgr2=0x7f1864199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641045a0 0x7f1864199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 -- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18641033b0 msgr2=0x7f1864199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18641033b0 0x7f1864199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 -- 172.21.15.6:0/758682829 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009900 con 0x7f1864104f60 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.536+0000 7f1863e00640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1864104f60 0x7f18641a09b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f185c037040 tx=0x7f185c032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.537+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c034e20 con 0x7f1864104f60 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.537+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18641a14a0 con 0x7f1864104f60 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.537+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f185c003e50 con 0x7f1864104f60 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.537+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c004800 con 0x7f1864104f60 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.537+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641a1920 con 0x7f1864104f60 2024-02-26T06:25:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.538+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186410a750 con 0x7f1864104f60 2024-02-26T06:25:23.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.540+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f185c00a040 con 0x7f1864104f60 2024-02-26T06:25:23.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.541+0000 7f185be00640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a20c0 0x7f18480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:23.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.541+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f185c0f05f0 con 0x7f1864104f60 2024-02-26T06:25:23.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.542+0000 7f1863400640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a20c0 0x7f18480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:23.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.542+0000 7f1863400640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a20c0 0x7f18480a4580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18540058c0 tx=0x7f1854005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:23.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.546+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f185c0b9670 con 0x7f1864104f60 2024-02-26T06:25:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: pgmap v320: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-02-26T06:25:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T06:25:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] up:active 2024-02-26T06:25:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T06:25:23.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-02-26T06:25:23.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-02-26T06:25:23.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-02-26T06:25:23.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/28633941' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:25:23.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:23 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:25:23.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: pgmap v320: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] up:active 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/28633941' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:25:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:25:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: pgmap v320: 97 pgs: 16 creating+peering, 81 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-02-26T06:25:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] up:active 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/28633941' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:25:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:23 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T06:25:23.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.773+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f186402ce80 con 0x7f18480a20c0 2024-02-26T06:25:23.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.806+0000 7f185be00640 1 -- 172.21.15.6:0/758682829 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f186402ce80 con 0x7f18480a20c0 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a20c0 msgr2=0x7f18480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a20c0 0x7f18480a4580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18540058c0 tx=0x7f1854005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1864104f60 msgr2=0x7f18641a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1864104f60 0x7f18641a09b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f185c037040 tx=0x7f185c032f80 comp rx=0 tx=0).stop 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 shutdown_connections 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a20c0 0x7f18480a4580 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1864104f60 0x7f18641a09b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18641045a0 0x7f1864199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 --2- 172.21.15.6:0/758682829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18641033b0 0x7f1864199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 >> 172.21.15.6:0/758682829 conn(0x7f18640fe940 msgr2=0x7f1864107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 shutdown_connections 2024-02-26T06:25:23.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:23.809+0000 7f186a000640 1 -- 172.21.15.6:0/758682829 wait complete. 2024-02-26T06:25:23.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 -- 172.21.15.6:0/2654899258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa0073be0 msgr2=0x7f2fa0074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 --2- 172.21.15.6:0/2654899258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa0073be0 0x7f2fa0074060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009930 tx=0x7f2f8c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 -- 172.21.15.6:0/2654899258 shutdown_connections 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 --2- 172.21.15.6:0/2654899258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa010d760 0x7f2fa010fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 --2- 172.21.15.6:0/2654899258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa0073be0 0x7f2fa0074060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 --2- 172.21.15.6:0/2654899258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa00751c0 0x7f2fa0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 -- 172.21.15.6:0/2654899258 >> 172.21.15.6:0/2654899258 conn(0x7f2fa00fc700 msgr2=0x7f2fa00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:24.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 -- 172.21.15.6:0/2654899258 shutdown_connections 2024-02-26T06:25:24.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.047+0000 7f2fa5200640 1 -- 172.21.15.6:0/2654899258 wait complete. 2024-02-26T06:25:24.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.048+0000 7f2fa5200640 1 Processor -- start 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.048+0000 7f2fa5200640 1 -- start start 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2fa5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 0x7f2fa019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2fa5200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa00751c0 0x7f2fa019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2fa5200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa010d760 0x7f2fa01a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2fa5200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa01a5380 con 0x7f2fa010d760 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2fa5200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa01a54f0 con 0x7f2fa00751c0 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2fa5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa01a56b0 con 0x7f2fa0073be0 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 0x7f2fa019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 0x7f2fa019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34468/0 (socket says 172.21.15.6:34468) 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 -- 172.21.15.6:0/1412980108 learned_addr learned my addr 172.21.15.6:0/1412980108 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9e800640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa010d760 0x7f2fa01a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9d400640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa00751c0 0x7f2fa019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 -- 172.21.15.6:0/1412980108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa00751c0 msgr2=0x7f2fa019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa00751c0 0x7f2fa019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 -- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa010d760 msgr2=0x7f2fa01a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa010d760 0x7f2fa01a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.049+0000 7f2f9de00640 1 -- 172.21.15.6:0/1412980108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c009560 con 0x7f2fa0073be0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.050+0000 7f2f9de00640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 0x7f2fa019d890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000e9f0 tx=0x7f2f9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.050+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90009700 con 0x7f2fa0073be0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.050+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f9000cd80 con 0x7f2fa0073be0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.050+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa01a5990 con 0x7f2fa0073be0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.050+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90018920 con 0x7f2fa0073be0 2024-02-26T06:25:24.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.050+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa01a5e70 con 0x7f2fa0073be0 2024-02-26T06:25:24.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.052+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa0103930 con 0x7f2fa0073be0 2024-02-26T06:25:24.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.054+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2f90018a80 con 0x7f2fa0073be0 2024-02-26T06:25:24.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.055+0000 7f2f96a00640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f840a20c0 0x7f2f840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.055+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2f90014070 con 0x7f2fa0073be0 2024-02-26T06:25:24.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.055+0000 7f2f9d400640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f840a20c0 0x7f2f840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.056+0000 7f2f9d400640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f840a20c0 0x7f2f840a4580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009900 tx=0x7f2f8c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:24.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.059+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f90052d20 con 0x7f2fa0073be0 2024-02-26T06:25:24.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:25:23.783+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:25:24.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:25:23.783+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:25:24.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:25:23.783+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:25:24.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:25:23.783+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:25:24.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:25:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:25:23.783+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:25:24.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.295+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa002ceb0 con 0x7f2f840a20c0 2024-02-26T06:25:24.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.330+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2fa002ceb0 con 0x7f2f840a20c0 2024-02-26T06:25:24.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.331+0000 7f2f96a00640 1 -- 172.21.15.6:0/1412980108 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f2f90097460 con 0x7f2fa0073be0 2024-02-26T06:25:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.334+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f840a20c0 msgr2=0x7f2f840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.334+0000 7f2fa5200640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f840a20c0 0x7f2f840a4580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009900 tx=0x7f2f8c02c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.334+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 msgr2=0x7f2fa019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.334+0000 7f2fa5200640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 0x7f2fa019d890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000e9f0 tx=0x7f2f9000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 shutdown_connections 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fa010d760 0x7f2fa01a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f840a20c0 0x7f2f840a4580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa00751c0 0x7f2fa019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 --2- 172.21.15.6:0/1412980108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa0073be0 0x7f2fa019d890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 >> 172.21.15.6:0/1412980108 conn(0x7f2fa00fc700 msgr2=0x7f2fa0100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 shutdown_connections 2024-02-26T06:25:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.335+0000 7f2fa5200640 1 -- 172.21.15.6:0/1412980108 wait complete. 2024-02-26T06:25:24.357 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-26T06:25:24.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 -- 172.21.15.6:0/4181752391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc654109370 msgr2=0x7fc65410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 --2- 172.21.15.6:0/4181752391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc654109370 0x7fc65410fc00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009be0 tx=0x7fc64c02bef0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 -- 172.21.15.6:0/4181752391 shutdown_connections 2024-02-26T06:25:24.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 --2- 172.21.15.6:0/4181752391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc654109370 0x7fc65410fc00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 --2- 172.21.15.6:0/4181752391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6541089b0 0x7fc654108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 --2- 172.21.15.6:0/4181752391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541077c0 0x7fc654107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 -- 172.21.15.6:0/4181752391 >> 172.21.15.6:0/4181752391 conn(0x7fc654076770 msgr2=0x7fc654078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:24.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 -- 172.21.15.6:0/4181752391 shutdown_connections 2024-02-26T06:25:24.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.559+0000 7fc65b400640 1 -- 172.21.15.6:0/4181752391 wait complete. 2024-02-26T06:25:24.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 Processor -- start 2024-02-26T06:25:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 -- start start 2024-02-26T06:25:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6541077c0 0x7fc65419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 0x7fc65419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc654109370 0x7fc6541a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541a52d0 con 0x7fc654109370 2024-02-26T06:25:24.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541a5440 con 0x7fc6541077c0 2024-02-26T06:25:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.560+0000 7fc65b400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541a5600 con 0x7fc6541089b0 2024-02-26T06:25:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 0x7fc65419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 0x7fc65419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34486/0 (socket says 172.21.15.6:34486) 2024-02-26T06:25:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc658a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6541077c0 0x7fc65419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 -- 172.21.15.6:0/3595191926 learned_addr learned my addr 172.21.15.6:0/3595191926 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc659400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc654109370 0x7fc6541a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 -- 172.21.15.6:0/3595191926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6541077c0 msgr2=0x7fc65419d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6541077c0 0x7fc65419d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 -- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc654109370 msgr2=0x7fc6541a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc654109370 0x7fc6541a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 -- 172.21.15.6:0/3595191926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009560 con 0x7fc6541089b0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc658a00640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6541077c0 0x7fc65419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.561+0000 7fc653e00640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 0x7fc65419dd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc64000e9f0 tx=0x7fc64000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.562+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640009700 con 0x7fc6541089b0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.562+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc64000cd80 con 0x7fc6541089b0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.562+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640018920 con 0x7fc6541089b0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.562+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c009840 con 0x7fc6541089b0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.562+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6541a5c10 con 0x7fc6541089b0 2024-02-26T06:25:24.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.563+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65410eae0 con 0x7fc6541089b0 2024-02-26T06:25:24.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.565+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fc640018a80 con 0x7fc6541089b0 2024-02-26T06:25:24.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.567+0000 7fc651600640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc6380a2150 0x7fc6380a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:24.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.567+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc640014070 con 0x7fc6541089b0 2024-02-26T06:25:24.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.567+0000 7fc658a00640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc6380a2150 0x7fc6380a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:24.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.568+0000 7fc658a00640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc6380a2150 0x7fc6380a4610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc644009530 tx=0x7fc644009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:24.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.571+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc640052d20 con 0x7fc6541089b0 2024-02-26T06:25:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T06:25:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:active 2024-02-26T06:25:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-26T06:25:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-02-26T06:25:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-02-26T06:25:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-02-26T06:25:24.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:24 smithi090 ceph-mon[35681]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:24.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T06:25:24.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:active 2024-02-26T06:25:24.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-26T06:25:24.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-02-26T06:25:24.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-02-26T06:25:24.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-02-26T06:25:24.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:24 smithi006 ceph-mon[31790]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T06:25:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:active 2024-02-26T06:25:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-26T06:25:24.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-02-26T06:25:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-02-26T06:25:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-02-26T06:25:24.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:24 smithi167 ceph-mon[36229]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:24.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.807+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc65402cee0 con 0x7fc6380a2150 2024-02-26T06:25:24.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.814+0000 7fc651600640 1 -- 172.21.15.6:0/3595191926 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc65402cee0 con 0x7fc6380a2150 2024-02-26T06:25:24.813 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/qa/sv_0/0f0fc196-62e8-4478-baa6-55f47b083796 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc6380a2150 msgr2=0x7fc6380a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc6380a2150 0x7fc6380a4610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc644009530 tx=0x7fc644009260 comp rx=0 tx=0).stop 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 msgr2=0x7fc65419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 0x7fc65419dd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc64000e9f0 tx=0x7fc64000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 shutdown_connections 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc654109370 0x7fc6541a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6541089b0 0x7fc65419dd20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc6380a2150 0x7fc6380a4610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 --2- 172.21.15.6:0/3595191926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6541077c0 0x7fc65419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 >> 172.21.15.6:0/3595191926 conn(0x7fc654076770 msgr2=0x7fc65410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:24.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 shutdown_connections 2024-02-26T06:25:24.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:24.817+0000 7fc65b400640 1 -- 172.21.15.6:0/3595191926 wait complete. 2024-02-26T06:25:24.836 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-26T06:25:25.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 -- 172.21.15.6:0/855767555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 msgr2=0x7f87a8109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 --2- 172.21.15.6:0/855767555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 0x7f87a8109100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009ca0 tx=0x7f87a002c100 comp rx=0 tx=0).stop 2024-02-26T06:25:25.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 -- 172.21.15.6:0/855767555 shutdown_connections 2024-02-26T06:25:25.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 --2- 172.21.15.6:0/855767555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 0x7f87a8109100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 --2- 172.21.15.6:0/855767555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a8104380 0x7f87a8106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 --2- 172.21.15.6:0/855767555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.053+0000 7f87aee00640 1 -- 172.21.15.6:0/855767555 >> 172.21.15.6:0/855767555 conn(0x7f87a80fb460 msgr2=0x7f87a80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.054+0000 7f87aee00640 1 -- 172.21.15.6:0/855767555 shutdown_connections 2024-02-26T06:25:25.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.054+0000 7f87aee00640 1 -- 172.21.15.6:0/855767555 wait complete. 2024-02-26T06:25:25.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.055+0000 7f87aee00640 1 Processor -- start 2024-02-26T06:25:25.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.055+0000 7f87aee00640 1 -- start start 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87aee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87aee00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a8104380 0x7f87a8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87aee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 0x7f87a81a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87aee00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81a0cf0 con 0x7f87a8104380 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87aee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81a0e60 con 0x7f87a8101a50 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87aee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81a1020 con 0x7f87a8106cb0 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87ada00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87ada00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34704/0 (socket says 172.21.15.6:34704) 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87ada00640 1 -- 172.21.15.6:0/3406336182 learned_addr learned my addr 172.21.15.6:0/3406336182 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87ad000640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a8104380 0x7f87a8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.056+0000 7f87ae400640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 0x7f87a81a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87ada00640 1 -- 172.21.15.6:0/3406336182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 msgr2=0x7f87a81a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87ada00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 0x7f87a81a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87ada00640 1 -- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a8104380 msgr2=0x7f87a8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87ada00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a8104380 0x7f87a8199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87ada00640 1 -- 172.21.15.6:0/3406336182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0009900 con 0x7f87a8101a50 2024-02-26T06:25:25.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87ada00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8199200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f879c00e9f0 tx=0x7f879c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c009720 con 0x7f87a8101a50 2024-02-26T06:25:25.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f879c00cd80 con 0x7f87a8101a50 2024-02-26T06:25:25.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a81a1300 con 0x7f87a8101a50 2024-02-26T06:25:25.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c010690 con 0x7f87a8101a50 2024-02-26T06:25:25.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.057+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a81a17c0 con 0x7f87a8101a50 2024-02-26T06:25:25.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.059+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a81087a0 con 0x7f87a8101a50 2024-02-26T06:25:25.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.060+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f879c0046c0 con 0x7f87a8101a50 2024-02-26T06:25:25.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.062+0000 7f87a6000640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87900a2150 0x7f87900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.062+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f879c014070 con 0x7f87a8101a50 2024-02-26T06:25:25.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.062+0000 7f87ad000640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87900a2150 0x7f87900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.063+0000 7f87ad000640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87900a2150 0x7f87900a4610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87a819a710 tx=0x7f879400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.066+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f879c0970d0 con 0x7f87a8101a50 2024-02-26T06:25:25.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.303+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f87a802ce40 con 0x7f87900a2150 2024-02-26T06:25:25.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.336+0000 7f87a6000640 1 -- 172.21.15.6:0/3406336182 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f87a802ce40 con 0x7f87900a2150 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87900a2150 msgr2=0x7f87900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87900a2150 0x7f87900a4610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87a819a710 tx=0x7f879400b390 comp rx=0 tx=0).stop 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 msgr2=0x7f87a8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8199200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f879c00e9f0 tx=0x7f879c00edf0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 shutdown_connections 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87a8106cb0 0x7f87a81a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a8104380 0x7f87a8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87900a2150 0x7f87900a4610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 --2- 172.21.15.6:0/3406336182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87a8101a50 0x7f87a8199200 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 >> 172.21.15.6:0/3406336182 conn(0x7f87a80fb460 msgr2=0x7f87a81010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 shutdown_connections 2024-02-26T06:25:25.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.339+0000 7f87aee00640 1 -- 172.21.15.6:0/3406336182 wait complete. 2024-02-26T06:25:25.358 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-26T06:25:25.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 -- 172.21.15.6:0/336793784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac104d90 msgr2=0x7f50ac10b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 --2- 172.21.15.6:0/336793784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac104d90 0x7f50ac10b620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009ca0 tx=0x7f50a402c0f0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 -- 172.21.15.6:0/336793784 shutdown_connections 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 --2- 172.21.15.6:0/336793784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac104d90 0x7f50ac10b620 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 --2- 172.21.15.6:0/336793784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac1043d0 0x7f50ac104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 --2- 172.21.15.6:0/336793784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac1031e0 0x7f50ac103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 -- 172.21.15.6:0/336793784 >> 172.21.15.6:0/336793784 conn(0x7f50ac0fe750 msgr2=0x7f50ac100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 -- 172.21.15.6:0/336793784 shutdown_connections 2024-02-26T06:25:25.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.586+0000 7f50b2600640 1 -- 172.21.15.6:0/336793784 wait complete. 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 Processor -- start 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 -- start start 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 0x7f50ac199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac1043d0 0x7f50ac1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac104d90 0x7f50ac1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a0d60 con 0x7f50ac1031e0 2024-02-26T06:25:25.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.587+0000 7f50b2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a0ed0 con 0x7f50ac104d90 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b2600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a1090 con 0x7f50ac1043d0 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 0x7f50ac199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac104d90 0x7f50ac1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 0x7f50ac199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39110/0 (socket says 172.21.15.6:39110) 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1200640 1 -- 172.21.15.6:0/4116724236 learned_addr learned my addr 172.21.15.6:0/4116724236 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50abe00640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac1043d0 0x7f50ac1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1c00640 1 -- 172.21.15.6:0/4116724236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac1043d0 msgr2=0x7f50ac1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1c00640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac1043d0 0x7f50ac1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1c00640 1 -- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 msgr2=0x7f50ac199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1c00640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 0x7f50ac199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1c00640 1 -- 172.21.15.6:0/4116724236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0009490 con 0x7f50ac104d90 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.588+0000 7f50b1200640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 0x7f50ac199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.589+0000 7f50b1c00640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac104d90 0x7f50ac1a0820 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50a402c700 tx=0x7f50a4034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.589+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a40329b0 con 0x7f50ac104d90 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.589+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50a4003e50 con 0x7f50ac104d90 2024-02-26T06:25:25.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.589+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a4009900 con 0x7f50ac104d90 2024-02-26T06:25:25.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.589+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4040450 con 0x7f50ac104d90 2024-02-26T06:25:25.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.589+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a1630 con 0x7f50ac104d90 2024-02-26T06:25:25.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.590+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac10a4a0 con 0x7f50ac104d90 2024-02-26T06:25:25.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.592+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f50a400a040 con 0x7f50ac104d90 2024-02-26T06:25:25.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.594+0000 7f50a9600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f50940a2150 0x7f50940a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:25.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.594+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50a40f0690 con 0x7f50ac104d90 2024-02-26T06:25:25.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.594+0000 7f50b1200640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f50940a2150 0x7f50940a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:25.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.595+0000 7f50b1200640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f50940a2150 0x7f50940a4610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50a00058c0 tx=0x7f50a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:25.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.596+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50a40b96c0 con 0x7f50ac104d90 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] up:active 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: Cluster is now healthy 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-02-26T06:25:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-02-26T06:25:25.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] up:active 2024-02-26T06:25:25.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:25:25.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:25 smithi090 ceph-mon[35681]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:25.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-26T06:25:25.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:25.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] up:active 2024-02-26T06:25:25.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-26T06:25:25.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: Cluster is now healthy 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] up:active 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:25:25.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:25 smithi167 ceph-mon[36229]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-26T06:25:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] up:active 2024-02-26T06:25:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-26T06:25:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T06:25:25.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: Cluster is now healthy 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] up:active 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:25:25.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:25 smithi006 ceph-mon[31790]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:25.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.827+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f50ac02ceb0 con 0x7f50940a2150 2024-02-26T06:25:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.834+0000 7f50a9600640 1 -- 172.21.15.6:0/4116724236 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f50ac02ceb0 con 0x7f50940a2150 2024-02-26T06:25:25.833 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018 2024-02-26T06:25:25.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f50940a2150 msgr2=0x7f50940a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f50940a2150 0x7f50940a4610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50a00058c0 tx=0x7f50a0005f90 comp rx=0 tx=0).stop 2024-02-26T06:25:25.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac104d90 msgr2=0x7f50ac1a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac104d90 0x7f50ac1a0820 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50a402c700 tx=0x7f50a4034a90 comp rx=0 tx=0).stop 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 shutdown_connections 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f50940a2150 0x7f50940a4610 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac104d90 0x7f50ac1a0820 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac1043d0 0x7f50ac1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 --2- 172.21.15.6:0/4116724236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac1031e0 0x7f50ac199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 >> 172.21.15.6:0/4116724236 conn(0x7f50ac0fe750 msgr2=0x7f50ac107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 shutdown_connections 2024-02-26T06:25:25.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:25.837+0000 7f50b2600640 1 -- 172.21.15.6:0/4116724236 wait complete. 2024-02-26T06:25:25.856 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 -- 172.21.15.6:0/893599299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104ff0 msgr2=0x7f6cc010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 --2- 172.21.15.6:0/893599299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104ff0 0x7f6cc010b880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009be0 tx=0x7f6cb802bef0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 -- 172.21.15.6:0/893599299 shutdown_connections 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 --2- 172.21.15.6:0/893599299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104ff0 0x7f6cc010b880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 --2- 172.21.15.6:0/893599299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104630 0x7f6cc0104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 --2- 172.21.15.6:0/893599299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 0x7f6cc0103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 -- 172.21.15.6:0/893599299 >> 172.21.15.6:0/893599299 conn(0x7f6cc00fe990 msgr2=0x7f6cc0100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 -- 172.21.15.6:0/893599299 shutdown_connections 2024-02-26T06:25:26.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.085+0000 7f6cc7200640 1 -- 172.21.15.6:0/893599299 wait complete. 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.086+0000 7f6cc7200640 1 Processor -- start 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.086+0000 7f6cc7200640 1 -- start start 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc7200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 0x7f6cc01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104630 0x7f6cc0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc7200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 0x7f6cc01a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc7200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc01a0ec0 con 0x7f6cc0104ff0 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc7200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc01a1030 con 0x7f6cc0103440 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc01a11f0 con 0x7f6cc0104630 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc5200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 0x7f6cc01a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc5200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 0x7f6cc01a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39124/0 (socket says 172.21.15.6:39124) 2024-02-26T06:25:26.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cbfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 0x7f6cc01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cc5200640 1 -- 172.21.15.6:0/3078853619 learned_addr learned my addr 172.21.15.6:0/3078853619 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cb7400640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104630 0x7f6cc0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cbfe00640 1 -- 172.21.15.6:0/3078853619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104630 msgr2=0x7f6cc0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.087+0000 7f6cbfe00640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104630 0x7f6cc0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbfe00640 1 -- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 msgr2=0x7f6cc01a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbfe00640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 0x7f6cc01a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbfe00640 1 -- 172.21.15.6:0/3078853619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb8009840 con 0x7f6cc0103440 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cc5200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 0x7f6cc01a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:26.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbfe00640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 0x7f6cc01993d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000e9c0 tx=0x7f6cb000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000ccc0 con 0x7f6cc0103440 2024-02-26T06:25:26.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc01a14d0 con 0x7f6cc0103440 2024-02-26T06:25:26.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb000ce20 con 0x7f6cc0103440 2024-02-26T06:25:26.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0010430 con 0x7f6cc0103440 2024-02-26T06:25:26.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.088+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc01a1a30 con 0x7f6cc0103440 2024-02-26T06:25:26.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.089+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc010a690 con 0x7f6cc0103440 2024-02-26T06:25:26.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.091+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f6cb0003f20 con 0x7f6cc0103440 2024-02-26T06:25:26.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.093+0000 7f6cbd600640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ca40a2150 0x7f6ca40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.093+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6cb0014070 con 0x7f6cc0103440 2024-02-26T06:25:26.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.093+0000 7f6cb7400640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ca40a2150 0x7f6ca40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.094+0000 7f6cb7400640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ca40a2150 0x7f6ca40a4610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6cc019a8e0 tx=0x7f6ca8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.097+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6cb0097020 con 0x7f6cc0103440 2024-02-26T06:25:26.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.347+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cc010ee00 con 0x7f6cc0103440 2024-02-26T06:25:26.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.348+0000 7f6cbd600640 1 -- 172.21.15.6:0/3078853619 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f6cb009a6e0 con 0x7f6cc0103440 2024-02-26T06:25:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:26.347 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:25:24.777181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:25:26.348 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.350+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ca40a2150 msgr2=0x7f6ca40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.350+0000 7f6cc7200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ca40a2150 0x7f6ca40a4610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6cc019a8e0 tx=0x7f6ca8009260 comp rx=0 tx=0).stop 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.350+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 msgr2=0x7f6cc01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.350+0000 7f6cc7200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 0x7f6cc01993d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000e9c0 tx=0x7f6cb000edc0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 shutdown_connections 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cc0104ff0 0x7f6cc01a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0104630 0x7f6cc0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6ca40a2150 0x7f6ca40a4610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 --2- 172.21.15.6:0/3078853619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc0103440 0x7f6cc01993d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 >> 172.21.15.6:0/3078853619 conn(0x7f6cc00fe990 msgr2=0x7f6cc0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 shutdown_connections 2024-02-26T06:25:26.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.351+0000 7f6cc7200640 1 -- 172.21.15.6:0/3078853619 wait complete. 2024-02-26T06:25:26.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:25:26.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 -- 172.21.15.6:0/2022242087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908104440 msgr2=0x7f2908106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 --2- 172.21.15.6:0/2022242087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908104440 0x7f2908106830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009900 tx=0x7f28f402bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 -- 172.21.15.6:0/2022242087 shutdown_connections 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 --2- 172.21.15.6:0/2022242087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2908106d70 0x7f29081091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 --2- 172.21.15.6:0/2022242087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908104440 0x7f2908106830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 --2- 172.21.15.6:0/2022242087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2908101b10 0x7f2908103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 -- 172.21.15.6:0/2022242087 >> 172.21.15.6:0/2022242087 conn(0x7f29080fb500 msgr2=0x7f29080fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 -- 172.21.15.6:0/2022242087 shutdown_connections 2024-02-26T06:25:26.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.581+0000 7f2907e00640 1 -- 172.21.15.6:0/2022242087 wait complete. 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.582+0000 7f2907e00640 1 Processor -- start 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.582+0000 7f2907e00640 1 -- start start 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.582+0000 7f2907e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 0x7f2908195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.582+0000 7f2907e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2908104440 0x7f29081961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.582+0000 7f2907e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2908106d70 0x7f2908192810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2907e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908192fa0 con 0x7f2908101b10 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2907e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908193110 con 0x7f2908104440 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2907e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081932d0 con 0x7f2908106d70 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 0x7f2908195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 0x7f2908195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39132/0 (socket says 172.21.15.6:39132) 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 -- 172.21.15.6:0/3038161436 learned_addr learned my addr 172.21.15.6:0/3038161436 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2907400640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2908106d70 0x7f2908192810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906000640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2908104440 0x7f29081961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 -- 172.21.15.6:0/3038161436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2908106d70 msgr2=0x7f2908192810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2908106d70 0x7f2908192810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 -- 172.21.15.6:0/3038161436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2908104440 msgr2=0x7f29081961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2908104440 0x7f29081961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.583+0000 7f2906a00640 1 -- 172.21.15.6:0/3038161436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4009560 con 0x7f2908101b10 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.584+0000 7f2906a00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 0x7f2908195ca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9f0 tx=0x7f28f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.584+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8009700 con 0x7f2908101b10 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.584+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f800cd80 con 0x7f2908101b10 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.584+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8018920 con 0x7f2908101b10 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.584+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29081a5830 con 0x7f2908101b10 2024-02-26T06:25:26.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.584+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29081a5d10 con 0x7f2908101b10 2024-02-26T06:25:26.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.585+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29081088c0 con 0x7f2908101b10 2024-02-26T06:25:26.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.587+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f28f8018ac0 con 0x7f2908101b10 2024-02-26T06:25:26.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.589+0000 7f290ca00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28ec0a2140 0x7f28ec0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:26.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.589+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f28f8014070 con 0x7f2908101b10 2024-02-26T06:25:26.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.589+0000 7f2906000640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28ec0a2140 0x7f28ec0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:26.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.590+0000 7f2906000640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28ec0a2140 0x7f28ec0a4600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f28f402c240 tx=0x7f28f402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.594+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28f8052d20 con 0x7f2908101b10 2024-02-26T06:25:26.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:26 smithi090 ceph-mon[35681]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:26.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:26 smithi090 ceph-mon[35681]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:26.668 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:26 smithi167 ceph-mon[36229]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:26.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:26 smithi167 ceph-mon[36229]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:26.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:26 smithi006 ceph-mon[31790]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:26.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:26 smithi006 ceph-mon[31790]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:25:26.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.796+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2908108330 con 0x7f2908101b10 2024-02-26T06:25:26.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.800+0000 7f290ca00640 1 -- 172.21.15.6:0/3038161436 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22114 (secure 0 0 0) 0x7f28f8097100 con 0x7f2908101b10 2024-02-26T06:25:26.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:26.799 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":85,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:25:19.176219+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:25:26.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.802+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28ec0a2140 msgr2=0x7f28ec0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.802+0000 7f2907e00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28ec0a2140 0x7f28ec0a4600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f28f402c240 tx=0x7f28f402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.802+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 msgr2=0x7f2908195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.802+0000 7f2907e00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 0x7f2908195ca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9f0 tx=0x7f28f800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 shutdown_connections 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2908106d70 0x7f2908192810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28ec0a2140 0x7f28ec0a4600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2908104440 0x7f29081961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 --2- 172.21.15.6:0/3038161436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2908101b10 0x7f2908195ca0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:26.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 >> 172.21.15.6:0/3038161436 conn(0x7f29080fb500 msgr2=0x7f2908101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:26.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 shutdown_connections 2024-02-26T06:25:26.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:26.803+0000 7f2907e00640 1 -- 172.21.15.6:0/3038161436 wait complete. 2024-02-26T06:25:26.820 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T06:25:26.820 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.045+0000 7fa7c3200640 1 -- 172.21.15.6:0/2686359530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4102270 msgr2=0x7fa7c41026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.045+0000 7fa7c3200640 1 --2- 172.21.15.6:0/2686359530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4102270 0x7fa7c41026f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009900 tx=0x7fa7ac02bc40 comp rx=0 tx=0).stop 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 -- 172.21.15.6:0/2686359530 shutdown_connections 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 --2- 172.21.15.6:0/2686359530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c4102c30 0x7fa7c41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 --2- 172.21.15.6:0/2686359530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4102270 0x7fa7c41026f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 --2- 172.21.15.6:0/2686359530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4101080 0x7fa7c41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 -- 172.21.15.6:0/2686359530 >> 172.21.15.6:0/2686359530 conn(0x7fa7c40fc610 msgr2=0x7fa7c40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 -- 172.21.15.6:0/2686359530 shutdown_connections 2024-02-26T06:25:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.046+0000 7fa7c3200640 1 -- 172.21.15.6:0/2686359530 wait complete. 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 Processor -- start 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 -- start start 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 0x7fa7c4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c4102270 0x7fa7c4197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4102c30 0x7fa7c419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c419ec20 con 0x7fa7c4102270 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c419ed90 con 0x7fa7c4102c30 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c419ef50 con 0x7fa7c4101080 2024-02-26T06:25:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 0x7fa7c4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 0x7fa7c4197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:55572/0 (socket says 172.21.15.6:55572) 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7c1e00640 1 -- 172.21.15.6:0/1853854336 learned_addr learned my addr 172.21.15.6:0/1853854336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.047+0000 7fa7b9400640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c4102270 0x7fa7c4197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c2800640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4102c30 0x7fa7c419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c1e00640 1 -- 172.21.15.6:0/1853854336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4102c30 msgr2=0x7fa7c419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c1e00640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4102c30 0x7fa7c419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c1e00640 1 -- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c4102270 msgr2=0x7fa7c4197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c1e00640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c4102270 0x7fa7c4197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c1e00640 1 -- 172.21.15.6:0/1853854336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac009560 con 0x7fa7c4101080 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c2800640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4102c30 0x7fa7c419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:27.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c1e00640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 0x7fa7c4197130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9c0 tx=0x7fa7b400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40096e0 con 0x7fa7c4101080 2024-02-26T06:25:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b400cd80 con 0x7fa7c4101080 2024-02-26T06:25:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c419f230 con 0x7fa7c4101080 2024-02-26T06:25:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.048+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4010430 con 0x7fa7c4101080 2024-02-26T06:25:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.049+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c419f710 con 0x7fa7c4101080 2024-02-26T06:25:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.050+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c4108340 con 0x7fa7c4101080 2024-02-26T06:25:27.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.052+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fa7b4003680 con 0x7fa7c4101080 2024-02-26T06:25:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.053+0000 7fa7bb400640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa7a80a21c0 0x7fa7a80a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.053+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7b4014070 con 0x7fa7c4101080 2024-02-26T06:25:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.053+0000 7fa7b9400640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa7a80a21c0 0x7fa7a80a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.054+0000 7fa7b9400640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa7a80a21c0 0x7fa7a80a4680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac02c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.055+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7b4097050 con 0x7fa7c4101080 2024-02-26T06:25:27.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.248+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7c402ce40 con 0x7fa7a80a21c0 2024-02-26T06:25:27.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.288+0000 7fa7bb400640 1 -- 172.21.15.6:0/1853854336 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338996 (secure 0 0 0) 0x7fa7c402ce40 con 0x7fa7a80a21c0 2024-02-26T06:25:27.289 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:27.295 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:25:27.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa7a80a21c0 msgr2=0x7fa7a80a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa7a80a21c0 0x7fa7a80a4680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac02c7f0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 msgr2=0x7fa7c4197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 0x7fa7c4197130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9c0 tx=0x7fa7b400ee80 comp rx=0 tx=0).stop 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 shutdown_connections 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa7a80a21c0 0x7fa7a80a4680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c4102c30 0x7fa7c419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c4102270 0x7fa7c4197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 --2- 172.21.15.6:0/1853854336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4101080 0x7fa7c4197130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.297+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 >> 172.21.15.6:0/1853854336 conn(0x7fa7c40fc610 msgr2=0x7fa7c4105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.298+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 shutdown_connections 2024-02-26T06:25:27.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.298+0000 7fa7c3200640 1 -- 172.21.15.6:0/1853854336 wait complete. 2024-02-26T06:25:27.324 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-02-26T06:25:26.108341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618310,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":157,"num_read_kb":136,"num_write":391,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378584,"kb_used_data":12784,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563240,"statfs":{"total":1151940427776,"available":1151552757760,"internally_reserved":0,"allocated":13090816,"data_stored":7096921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27942,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":210,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003633"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516169+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:25:19.516169+0000","last_peered":"2024-02-26T06:25:19.516169+0000","last_clean":"2024-02-26T06:25:19.516169+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:25:19.516169+0000","last_undegraded":"2024-02-26T06:25:19.516169+0000","last_fullsized":"2024-02-26T06:25:19.516169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182322+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:25:19.182322+0000","last_peered":"2024-02-26T06:25:19.182322+0000","last_clean":"2024-02-26T06:25:19.182322+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:25:19.182322+0000","last_undegraded":"2024-02-26T06:25:19.182322+0000","last_fullsized":"2024-02-26T06:25:19.182322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182572+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:25:19.182572+0000","last_peered":"2024-02-26T06:25:19.182572+0000","last_clean":"2024-02-26T06:25:19.182572+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:25:19.182572+0000","last_undegraded":"2024-02-26T06:25:19.182572+0000","last_fullsized":"2024-02-26T06:25:19.182572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516185+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:25:19.516185+0000","last_peered":"2024-02-26T06:25:19.516185+0000","last_clean":"2024-02-26T06:25:19.516185+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:25:19.516185+0000","last_undegraded":"2024-02-26T06:25:19.516185+0000","last_fullsized":"2024-02-26T06:25:19.516185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.371104+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:25:19.371104+0000","last_peered":"2024-02-26T06:25:19.371104+0000","last_clean":"2024-02-26T06:25:19.371104+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:25:19.371104+0000","last_undegraded":"2024-02-26T06:25:19.371104+0000","last_fullsized":"2024-02-26T06:25:19.371104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516246+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:25:19.516246+0000","last_peered":"2024-02-26T06:25:19.516246+0000","last_clean":"2024-02-26T06:25:19.516246+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:25:19.516246+0000","last_undegraded":"2024-02-26T06:25:19.516246+0000","last_fullsized":"2024-02-26T06:25:19.516246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515358+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:25:19.515358+0000","last_peered":"2024-02-26T06:25:19.515358+0000","last_clean":"2024-02-26T06:25:19.515358+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:25:19.515358+0000","last_undegraded":"2024-02-26T06:25:19.515358+0000","last_fullsized":"2024-02-26T06:25:19.515358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.518444+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:25:19.518444+0000","last_peered":"2024-02-26T06:25:19.518444+0000","last_clean":"2024-02-26T06:25:19.518444+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:25:19.518444+0000","last_undegraded":"2024-02-26T06:25:19.518444+0000","last_fullsized":"2024-02-26T06:25:19.518444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181756+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:25:19.181756+0000","last_peered":"2024-02-26T06:25:19.181756+0000","last_clean":"2024-02-26T06:25:19.181756+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:25:19.181756+0000","last_undegraded":"2024-02-26T06:25:19.181756+0000","last_fullsized":"2024-02-26T06:25:19.181756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515786+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:25:19.515786+0000","last_peered":"2024-02-26T06:25:19.515786+0000","last_clean":"2024-02-26T06:25:19.515786+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:25:19.515786+0000","last_undegraded":"2024-02-26T06:25:19.515786+0000","last_fullsized":"2024-02-26T06:25:19.515786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516491+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:25:19.516491+0000","last_peered":"2024-02-26T06:25:19.516491+0000","last_clean":"2024-02-26T06:25:19.516491+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:25:19.516491+0000","last_undegraded":"2024-02-26T06:25:19.516491+0000","last_fullsized":"2024-02-26T06:25:19.516491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516142+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:25:19.516142+0000","last_peered":"2024-02-26T06:25:19.516142+0000","last_clean":"2024-02-26T06:25:19.516142+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:25:19.516142+0000","last_undegraded":"2024-02-26T06:25:19.516142+0000","last_fullsized":"2024-02-26T06:25:19.516142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.213993+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:25:19.213993+0000","last_peered":"2024-02-26T06:25:19.213993+0000","last_clean":"2024-02-26T06:25:19.213993+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:25:19.213993+0000","last_undegraded":"2024-02-26T06:25:19.213993+0000","last_fullsized":"2024-02-26T06:25:19.213993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181531+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:25:19.181531+0000","last_peered":"2024-02-26T06:25:19.181531+0000","last_clean":"2024-02-26T06:25:19.181531+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:25:19.181531+0000","last_undegraded":"2024-02-26T06:25:19.181531+0000","last_fullsized":"2024-02-26T06:25:19.181531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181587+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:25:19.181587+0000","last_peered":"2024-02-26T06:25:19.181587+0000","last_clean":"2024-02-26T06:25:19.181587+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:25:19.181587+0000","last_undegraded":"2024-02-26T06:25:19.181587+0000","last_fullsized":"2024-02-26T06:25:19.181587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181308+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:25:19.181308+0000","last_peered":"2024-02-26T06:25:19.181308+0000","last_clean":"2024-02-26T06:25:19.181308+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:25:19.181308+0000","last_undegraded":"2024-02-26T06:25:19.181308+0000","last_fullsized":"2024-02-26T06:25:19.181308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516298+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:25:19.516298+0000","last_peered":"2024-02-26T06:25:19.516298+0000","last_clean":"2024-02-26T06:25:19.516298+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:25:19.516298+0000","last_undegraded":"2024-02-26T06:25:19.516298+0000","last_fullsized":"2024-02-26T06:25:19.516298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.519897+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:25:19.519897+0000","last_peered":"2024-02-26T06:25:19.519897+0000","last_clean":"2024-02-26T06:25:19.519897+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:25:19.519897+0000","last_undegraded":"2024-02-26T06:25:19.519897+0000","last_fullsized":"2024-02-26T06:25:19.519897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181756+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:25:19.181756+0000","last_peered":"2024-02-26T06:25:19.181756+0000","last_clean":"2024-02-26T06:25:19.181756+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:25:19.181756+0000","last_undegraded":"2024-02-26T06:25:19.181756+0000","last_fullsized":"2024-02-26T06:25:19.181756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.789692+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:25:24.789692+0000","last_peered":"2024-02-26T06:25:24.789692+0000","last_clean":"2024-02-26T06:25:24.789692+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:25:24.789692+0000","last_undegraded":"2024-02-26T06:25:24.789692+0000","last_fullsized":"2024-02-26T06:25:24.789692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520369+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:25:19.520369+0000","last_peered":"2024-02-26T06:25:19.520369+0000","last_clean":"2024-02-26T06:25:19.520369+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:25:19.520369+0000","last_undegraded":"2024-02-26T06:25:19.520369+0000","last_fullsized":"2024-02-26T06:25:19.520369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520788+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:25:19.520788+0000","last_peered":"2024-02-26T06:25:19.520788+0000","last_clean":"2024-02-26T06:25:19.520788+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:25:19.520788+0000","last_undegraded":"2024-02-26T06:25:19.520788+0000","last_fullsized":"2024-02-26T06:25:19.520788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182680+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:25:19.182680+0000","last_peered":"2024-02-26T06:25:19.182680+0000","last_clean":"2024-02-26T06:25:19.182680+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:25:19.182680+0000","last_undegraded":"2024-02-26T06:25:19.182680+0000","last_fullsized":"2024-02-26T06:25:19.182680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'11","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.347149+0000","last_change":"2024-02-26T06:25:17.202246+0000","last_active":"2024-02-26T06:25:24.347149+0000","last_peered":"2024-02-26T06:25:24.347149+0000","last_clean":"2024-02-26T06:25:24.347149+0000","last_became_active":"2024-02-26T06:25:17.201861+0000","last_became_peered":"2024-02-26T06:25:17.201861+0000","last_unstale":"2024-02-26T06:25:24.347149+0000","last_undegraded":"2024-02-26T06:25:24.347149+0000","last_fullsized":"2024-02-26T06:25:24.347149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:14.968655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181365+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:25:19.181365+0000","last_peered":"2024-02-26T06:25:19.181365+0000","last_clean":"2024-02-26T06:25:19.181365+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:25:19.181365+0000","last_undegraded":"2024-02-26T06:25:19.181365+0000","last_fullsized":"2024-02-26T06:25:19.181365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.346132+0000","last_change":"2024-02-26T06:25:17.200321+0000","last_active":"2024-02-26T06:25:24.346132+0000","last_peered":"2024-02-26T06:25:24.346132+0000","last_clean":"2024-02-26T06:25:24.346132+0000","last_became_active":"2024-02-26T06:25:17.199957+0000","last_became_peered":"2024-02-26T06:25:17.199957+0000","last_unstale":"2024-02-26T06:25:24.346132+0000","last_undegraded":"2024-02-26T06:25:24.346132+0000","last_fullsized":"2024-02-26T06:25:24.346132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:20.743399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516570+0000","last_change":"2024-02-26T06:25:18.205619+0000","last_active":"2024-02-26T06:25:19.516570+0000","last_peered":"2024-02-26T06:25:19.516570+0000","last_clean":"2024-02-26T06:25:19.516570+0000","last_became_active":"2024-02-26T06:25:18.205278+0000","last_became_peered":"2024-02-26T06:25:18.205278+0000","last_unstale":"2024-02-26T06:25:19.516570+0000","last_undegraded":"2024-02-26T06:25:19.516570+0000","last_fullsized":"2024-02-26T06:25:19.516570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:48.102668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.287791+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:25:21.287791+0000","last_peered":"2024-02-26T06:25:21.287791+0000","last_clean":"2024-02-26T06:25:21.287791+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:25:21.287791+0000","last_undegraded":"2024-02-26T06:25:21.287791+0000","last_fullsized":"2024-02-26T06:25:21.287791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.371108+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:25:19.371108+0000","last_peered":"2024-02-26T06:25:19.371108+0000","last_clean":"2024-02-26T06:25:19.371108+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:25:19.371108+0000","last_undegraded":"2024-02-26T06:25:19.371108+0000","last_fullsized":"2024-02-26T06:25:19.371108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.317523+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:25:23.317523+0000","last_peered":"2024-02-26T06:25:23.317523+0000","last_clean":"2024-02-26T06:25:23.317523+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:25:23.317523+0000","last_undegraded":"2024-02-26T06:25:23.317523+0000","last_fullsized":"2024-02-26T06:25:23.317523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516419+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:25:19.516419+0000","last_peered":"2024-02-26T06:25:19.516419+0000","last_clean":"2024-02-26T06:25:19.516419+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:25:19.516419+0000","last_undegraded":"2024-02-26T06:25:19.516419+0000","last_fullsized":"2024-02-26T06:25:19.516419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:22.304433+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:25:22.304433+0000","last_peered":"2024-02-26T06:25:22.304433+0000","last_clean":"2024-02-26T06:25:22.304433+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:25:22.304433+0000","last_undegraded":"2024-02-26T06:25:22.304433+0000","last_fullsized":"2024-02-26T06:25:22.304433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.518368+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:25:19.518368+0000","last_peered":"2024-02-26T06:25:19.518368+0000","last_clean":"2024-02-26T06:25:19.518368+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:25:19.518368+0000","last_undegraded":"2024-02-26T06:25:19.518368+0000","last_fullsized":"2024-02-26T06:25:19.518368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.305225+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:25:23.305225+0000","last_peered":"2024-02-26T06:25:23.305225+0000","last_clean":"2024-02-26T06:25:23.305225+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:25:23.305225+0000","last_undegraded":"2024-02-26T06:25:23.305225+0000","last_fullsized":"2024-02-26T06:25:23.305225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.519026+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:25:19.519026+0000","last_peered":"2024-02-26T06:25:19.519026+0000","last_clean":"2024-02-26T06:25:19.519026+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:25:19.519026+0000","last_undegraded":"2024-02-26T06:25:19.519026+0000","last_fullsized":"2024-02-26T06:25:19.519026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.287226+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:25:21.287226+0000","last_peered":"2024-02-26T06:25:21.287226+0000","last_clean":"2024-02-26T06:25:21.287226+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:25:21.287226+0000","last_undegraded":"2024-02-26T06:25:21.287226+0000","last_fullsized":"2024-02-26T06:25:21.287226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181723+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:25:19.181723+0000","last_peered":"2024-02-26T06:25:19.181723+0000","last_clean":"2024-02-26T06:25:19.181723+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:25:19.181723+0000","last_undegraded":"2024-02-26T06:25:19.181723+0000","last_fullsized":"2024-02-26T06:25:19.181723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.345584+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:25:24.345584+0000","last_peered":"2024-02-26T06:25:24.345584+0000","last_clean":"2024-02-26T06:25:24.345584+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:25:24.345584+0000","last_undegraded":"2024-02-26T06:25:24.345584+0000","last_fullsized":"2024-02-26T06:25:24.345584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182851+0000","last_change":"2024-02-26T06:25:18.227237+0000","last_active":"2024-02-26T06:25:19.182851+0000","last_peered":"2024-02-26T06:25:19.182851+0000","last_clean":"2024-02-26T06:25:19.182851+0000","last_became_active":"2024-02-26T06:25:18.226821+0000","last_became_peered":"2024-02-26T06:25:18.226821+0000","last_unstale":"2024-02-26T06:25:19.182851+0000","last_undegraded":"2024-02-26T06:25:19.182851+0000","last_fullsized":"2024-02-26T06:25:19.182851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:23.200468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.287695+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:25:21.287695+0000","last_peered":"2024-02-26T06:25:21.287695+0000","last_clean":"2024-02-26T06:25:21.287695+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:25:21.287695+0000","last_undegraded":"2024-02-26T06:25:21.287695+0000","last_fullsized":"2024-02-26T06:25:21.287695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181545+0000","last_change":"2024-02-26T06:25:18.202144+0000","last_active":"2024-02-26T06:25:19.181545+0000","last_peered":"2024-02-26T06:25:19.181545+0000","last_clean":"2024-02-26T06:25:19.181545+0000","last_became_active":"2024-02-26T06:25:18.201824+0000","last_became_peered":"2024-02-26T06:25:18.201824+0000","last_unstale":"2024-02-26T06:25:19.181545+0000","last_undegraded":"2024-02-26T06:25:19.181545+0000","last_fullsized":"2024-02-26T06:25:19.181545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:31.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'17","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.346374+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:25:24.346374+0000","last_peered":"2024-02-26T06:25:24.346374+0000","last_clean":"2024-02-26T06:25:24.346374+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:25:24.346374+0000","last_undegraded":"2024-02-26T06:25:24.346374+0000","last_fullsized":"2024-02-26T06:25:24.346374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.371194+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:25:19.371194+0000","last_peered":"2024-02-26T06:25:19.371194+0000","last_clean":"2024-02-26T06:25:19.371194+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:25:19.371194+0000","last_undegraded":"2024-02-26T06:25:19.371194+0000","last_fullsized":"2024-02-26T06:25:19.371194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.787867+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:25:24.787867+0000","last_peered":"2024-02-26T06:25:24.787867+0000","last_clean":"2024-02-26T06:25:24.787867+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:25:24.787867+0000","last_undegraded":"2024-02-26T06:25:24.787867+0000","last_fullsized":"2024-02-26T06:25:24.787867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.214388+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:25:19.214388+0000","last_peered":"2024-02-26T06:25:19.214388+0000","last_clean":"2024-02-26T06:25:19.214388+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:25:19.214388+0000","last_undegraded":"2024-02-26T06:25:19.214388+0000","last_fullsized":"2024-02-26T06:25:19.214388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.306753+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:25:23.306753+0000","last_peered":"2024-02-26T06:25:23.306753+0000","last_clean":"2024-02-26T06:25:23.306753+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:25:23.306753+0000","last_undegraded":"2024-02-26T06:25:23.306753+0000","last_fullsized":"2024-02-26T06:25:23.306753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516655+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:25:19.516655+0000","last_peered":"2024-02-26T06:25:19.516655+0000","last_clean":"2024-02-26T06:25:19.516655+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:25:19.516655+0000","last_undegraded":"2024-02-26T06:25:19.516655+0000","last_fullsized":"2024-02-26T06:25:19.516655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.790323+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:25:24.790323+0000","last_peered":"2024-02-26T06:25:24.790323+0000","last_clean":"2024-02-26T06:25:24.790323+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:25:24.790323+0000","last_undegraded":"2024-02-26T06:25:24.790323+0000","last_fullsized":"2024-02-26T06:25:24.790323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516046+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:25:19.516046+0000","last_peered":"2024-02-26T06:25:19.516046+0000","last_clean":"2024-02-26T06:25:19.516046+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:25:19.516046+0000","last_undegraded":"2024-02-26T06:25:19.516046+0000","last_fullsized":"2024-02-26T06:25:19.516046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.314763+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:25:23.314763+0000","last_peered":"2024-02-26T06:25:23.314763+0000","last_clean":"2024-02-26T06:25:23.314763+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:25:23.314763+0000","last_undegraded":"2024-02-26T06:25:23.314763+0000","last_fullsized":"2024-02-26T06:25:23.314763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515611+0000","last_change":"2024-02-26T06:25:18.204602+0000","last_active":"2024-02-26T06:25:19.515611+0000","last_peered":"2024-02-26T06:25:19.515611+0000","last_clean":"2024-02-26T06:25:19.515611+0000","last_became_active":"2024-02-26T06:25:18.204258+0000","last_became_peered":"2024-02-26T06:25:18.204258+0000","last_unstale":"2024-02-26T06:25:19.515611+0000","last_undegraded":"2024-02-26T06:25:19.515611+0000","last_fullsized":"2024-02-26T06:25:19.515611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:59:45.631076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.749660+0000","last_change":"2024-02-26T06:25:17.200714+0000","last_active":"2024-02-26T06:25:19.749660+0000","last_peered":"2024-02-26T06:25:19.749660+0000","last_clean":"2024-02-26T06:25:19.749660+0000","last_became_active":"2024-02-26T06:25:17.200367+0000","last_became_peered":"2024-02-26T06:25:17.200367+0000","last_unstale":"2024-02-26T06:25:19.749660+0000","last_undegraded":"2024-02-26T06:25:19.749660+0000","last_fullsized":"2024-02-26T06:25:19.749660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:15.725139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516187+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:25:19.516187+0000","last_peered":"2024-02-26T06:25:19.516187+0000","last_clean":"2024-02-26T06:25:19.516187+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:25:19.516187+0000","last_undegraded":"2024-02-26T06:25:19.516187+0000","last_fullsized":"2024-02-26T06:25:19.516187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.992901+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:25:19.992901+0000","last_peered":"2024-02-26T06:25:19.992901+0000","last_clean":"2024-02-26T06:25:19.992901+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:25:19.992901+0000","last_undegraded":"2024-02-26T06:25:19.992901+0000","last_fullsized":"2024-02-26T06:25:19.992901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515868+0000","last_change":"2024-02-26T06:25:18.212398+0000","last_active":"2024-02-26T06:25:19.515868+0000","last_peered":"2024-02-26T06:25:19.515868+0000","last_clean":"2024-02-26T06:25:19.515868+0000","last_became_active":"2024-02-26T06:25:18.212021+0000","last_became_peered":"2024-02-26T06:25:18.212021+0000","last_unstale":"2024-02-26T06:25:19.515868+0000","last_undegraded":"2024-02-26T06:25:19.515868+0000","last_fullsized":"2024-02-26T06:25:19.515868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:21.364023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.791911+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:25:24.791911+0000","last_peered":"2024-02-26T06:25:24.791911+0000","last_clean":"2024-02-26T06:25:24.791911+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:25:24.791911+0000","last_undegraded":"2024-02-26T06:25:24.791911+0000","last_fullsized":"2024-02-26T06:25:24.791911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516899+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:25:19.516899+0000","last_peered":"2024-02-26T06:25:19.516899+0000","last_clean":"2024-02-26T06:25:19.516899+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:25:19.516899+0000","last_undegraded":"2024-02-26T06:25:19.516899+0000","last_fullsized":"2024-02-26T06:25:19.516899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515694+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:25:19.515694+0000","last_peered":"2024-02-26T06:25:19.515694+0000","last_clean":"2024-02-26T06:25:19.515694+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:25:19.515694+0000","last_undegraded":"2024-02-26T06:25:19.515694+0000","last_fullsized":"2024-02-26T06:25:19.515694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.787855+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:25:24.787855+0000","last_peered":"2024-02-26T06:25:24.787855+0000","last_clean":"2024-02-26T06:25:24.787855+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:25:24.787855+0000","last_undegraded":"2024-02-26T06:25:24.787855+0000","last_fullsized":"2024-02-26T06:25:24.787855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.519792+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:25:19.519792+0000","last_peered":"2024-02-26T06:25:19.519792+0000","last_clean":"2024-02-26T06:25:19.519792+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:25:19.519792+0000","last_undegraded":"2024-02-26T06:25:19.519792+0000","last_fullsized":"2024-02-26T06:25:19.519792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.787693+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:25:24.787693+0000","last_peered":"2024-02-26T06:25:24.787693+0000","last_clean":"2024-02-26T06:25:24.787693+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:25:24.787693+0000","last_undegraded":"2024-02-26T06:25:24.787693+0000","last_fullsized":"2024-02-26T06:25:24.787693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515988+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:25:19.515988+0000","last_peered":"2024-02-26T06:25:19.515988+0000","last_clean":"2024-02-26T06:25:19.515988+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:25:19.515988+0000","last_undegraded":"2024-02-26T06:25:19.515988+0000","last_fullsized":"2024-02-26T06:25:19.515988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.213536+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:25:19.213536+0000","last_peered":"2024-02-26T06:25:19.213536+0000","last_clean":"2024-02-26T06:25:19.213536+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:25:19.213536+0000","last_undegraded":"2024-02-26T06:25:19.213536+0000","last_fullsized":"2024-02-26T06:25:19.213536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:22.295320+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:25:22.295320+0000","last_peered":"2024-02-26T06:25:22.295320+0000","last_clean":"2024-02-26T06:25:22.295320+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:25:22.295320+0000","last_undegraded":"2024-02-26T06:25:22.295320+0000","last_fullsized":"2024-02-26T06:25:22.295320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.314887+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:25:23.314887+0000","last_peered":"2024-02-26T06:25:23.314887+0000","last_clean":"2024-02-26T06:25:23.314887+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:25:23.314887+0000","last_undegraded":"2024-02-26T06:25:23.314887+0000","last_fullsized":"2024-02-26T06:25:23.314887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.214421+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:25:19.214421+0000","last_peered":"2024-02-26T06:25:19.214421+0000","last_clean":"2024-02-26T06:25:19.214421+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:25:19.214421+0000","last_undegraded":"2024-02-26T06:25:19.214421+0000","last_fullsized":"2024-02-26T06:25:19.214421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.317512+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:25:23.317512+0000","last_peered":"2024-02-26T06:25:23.317512+0000","last_clean":"2024-02-26T06:25:23.317512+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:25:23.317512+0000","last_undegraded":"2024-02-26T06:25:23.317512+0000","last_fullsized":"2024-02-26T06:25:23.317512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520757+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:25:19.520757+0000","last_peered":"2024-02-26T06:25:19.520757+0000","last_clean":"2024-02-26T06:25:19.520757+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:25:19.520757+0000","last_undegraded":"2024-02-26T06:25:19.520757+0000","last_fullsized":"2024-02-26T06:25:19.520757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.349057+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:25:24.349057+0000","last_peered":"2024-02-26T06:25:24.349057+0000","last_clean":"2024-02-26T06:25:24.349057+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:25:24.349057+0000","last_undegraded":"2024-02-26T06:25:24.349057+0000","last_fullsized":"2024-02-26T06:25:24.349057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182424+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:25:19.182424+0000","last_peered":"2024-02-26T06:25:19.182424+0000","last_clean":"2024-02-26T06:25:19.182424+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:25:19.182424+0000","last_undegraded":"2024-02-26T06:25:19.182424+0000","last_fullsized":"2024-02-26T06:25:19.182424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.286690+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:25:21.286690+0000","last_peered":"2024-02-26T06:25:21.286690+0000","last_clean":"2024-02-26T06:25:21.286690+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:25:21.286690+0000","last_undegraded":"2024-02-26T06:25:21.286690+0000","last_fullsized":"2024-02-26T06:25:21.286690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.370895+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:25:19.370895+0000","last_peered":"2024-02-26T06:25:19.370895+0000","last_clean":"2024-02-26T06:25:19.370895+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:25:19.370895+0000","last_undegraded":"2024-02-26T06:25:19.370895+0000","last_fullsized":"2024-02-26T06:25:19.370895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.289515+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:25:21.289515+0000","last_peered":"2024-02-26T06:25:21.289515+0000","last_clean":"2024-02-26T06:25:21.289515+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:25:21.289515+0000","last_undegraded":"2024-02-26T06:25:21.289515+0000","last_fullsized":"2024-02-26T06:25:21.289515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515778+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:25:19.515778+0000","last_peered":"2024-02-26T06:25:19.515778+0000","last_clean":"2024-02-26T06:25:19.515778+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:25:19.515778+0000","last_undegraded":"2024-02-26T06:25:19.515778+0000","last_fullsized":"2024-02-26T06:25:19.515778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.312339+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:25:23.312339+0000","last_peered":"2024-02-26T06:25:23.312339+0000","last_clean":"2024-02-26T06:25:23.312339+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:25:23.312339+0000","last_undegraded":"2024-02-26T06:25:23.312339+0000","last_fullsized":"2024-02-26T06:25:23.312339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520099+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:25:19.520099+0000","last_peered":"2024-02-26T06:25:19.520099+0000","last_clean":"2024-02-26T06:25:19.520099+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:25:19.520099+0000","last_undegraded":"2024-02-26T06:25:19.520099+0000","last_fullsized":"2024-02-26T06:25:19.520099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.286487+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:25:21.286487+0000","last_peered":"2024-02-26T06:25:21.286487+0000","last_clean":"2024-02-26T06:25:21.286487+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:25:21.286487+0000","last_undegraded":"2024-02-26T06:25:21.286487+0000","last_fullsized":"2024-02-26T06:25:21.286487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516881+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:25:19.516881+0000","last_peered":"2024-02-26T06:25:19.516881+0000","last_clean":"2024-02-26T06:25:19.516881+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:25:19.516881+0000","last_undegraded":"2024-02-26T06:25:19.516881+0000","last_fullsized":"2024-02-26T06:25:19.516881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.792620+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:25:24.792620+0000","last_peered":"2024-02-26T06:25:24.792620+0000","last_clean":"2024-02-26T06:25:24.792620+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:25:24.792620+0000","last_undegraded":"2024-02-26T06:25:24.792620+0000","last_fullsized":"2024-02-26T06:25:24.792620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516588+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:25:19.516588+0000","last_peered":"2024-02-26T06:25:19.516588+0000","last_clean":"2024-02-26T06:25:19.516588+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:25:19.516588+0000","last_undegraded":"2024-02-26T06:25:19.516588+0000","last_fullsized":"2024-02-26T06:25:19.516588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.308763+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:25:23.308763+0000","last_peered":"2024-02-26T06:25:23.308763+0000","last_clean":"2024-02-26T06:25:23.308763+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:25:23.308763+0000","last_undegraded":"2024-02-26T06:25:23.308763+0000","last_fullsized":"2024-02-26T06:25:23.308763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:20.479971+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:25:20.479971+0000","last_peered":"2024-02-26T06:25:20.479971+0000","last_clean":"2024-02-26T06:25:20.479971+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:25:20.479971+0000","last_undegraded":"2024-02-26T06:25:20.479971+0000","last_fullsized":"2024-02-26T06:25:20.479971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.348595+0000","last_change":"2024-02-26T06:25:17.217008+0000","last_active":"2024-02-26T06:25:24.348595+0000","last_peered":"2024-02-26T06:25:24.348595+0000","last_clean":"2024-02-26T06:25:24.348595+0000","last_became_active":"2024-02-26T06:25:17.214955+0000","last_became_peered":"2024-02-26T06:25:17.214955+0000","last_unstale":"2024-02-26T06:25:24.348595+0000","last_undegraded":"2024-02-26T06:25:24.348595+0000","last_fullsized":"2024-02-26T06:25:24.348595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:58:50.627132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516724+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:25:19.516724+0000","last_peered":"2024-02-26T06:25:19.516724+0000","last_clean":"2024-02-26T06:25:19.516724+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:25:19.516724+0000","last_undegraded":"2024-02-26T06:25:19.516724+0000","last_fullsized":"2024-02-26T06:25:19.516724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515914+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:25:19.515914+0000","last_peered":"2024-02-26T06:25:19.515914+0000","last_clean":"2024-02-26T06:25:19.515914+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:25:19.515914+0000","last_undegraded":"2024-02-26T06:25:19.515914+0000","last_fullsized":"2024-02-26T06:25:19.515914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181474+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:25:19.181474+0000","last_peered":"2024-02-26T06:25:19.181474+0000","last_clean":"2024-02-26T06:25:19.181474+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:25:19.181474+0000","last_undegraded":"2024-02-26T06:25:19.181474+0000","last_fullsized":"2024-02-26T06:25:19.181474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181513+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:25:19.181513+0000","last_peered":"2024-02-26T06:25:19.181513+0000","last_clean":"2024-02-26T06:25:19.181513+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:25:19.181513+0000","last_undegraded":"2024-02-26T06:25:19.181513+0000","last_fullsized":"2024-02-26T06:25:19.181513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181971+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:25:19.181971+0000","last_peered":"2024-02-26T06:25:19.181971+0000","last_clean":"2024-02-26T06:25:19.181971+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:25:19.181971+0000","last_undegraded":"2024-02-26T06:25:19.181971+0000","last_fullsized":"2024-02-26T06:25:19.181971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.214402+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:25:19.214402+0000","last_peered":"2024-02-26T06:25:19.214402+0000","last_clean":"2024-02-26T06:25:19.214402+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:25:19.214402+0000","last_undegraded":"2024-02-26T06:25:19.214402+0000","last_fullsized":"2024-02-26T06:25:19.214402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515978+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:25:19.515978+0000","last_peered":"2024-02-26T06:25:19.515978+0000","last_clean":"2024-02-26T06:25:19.515978+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:25:19.515978+0000","last_undegraded":"2024-02-26T06:25:19.515978+0000","last_fullsized":"2024-02-26T06:25:19.515978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181914+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:25:19.181914+0000","last_peered":"2024-02-26T06:25:19.181914+0000","last_clean":"2024-02-26T06:25:19.181914+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:25:19.181914+0000","last_undegraded":"2024-02-26T06:25:19.181914+0000","last_fullsized":"2024-02-26T06:25:19.181914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520579+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:25:19.520579+0000","last_peered":"2024-02-26T06:25:19.520579+0000","last_clean":"2024-02-26T06:25:19.520579+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:25:19.520579+0000","last_undegraded":"2024-02-26T06:25:19.520579+0000","last_fullsized":"2024-02-26T06:25:19.520579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515449+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:25:19.515449+0000","last_peered":"2024-02-26T06:25:19.515449+0000","last_clean":"2024-02-26T06:25:19.515449+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:25:19.515449+0000","last_undegraded":"2024-02-26T06:25:19.515449+0000","last_fullsized":"2024-02-26T06:25:19.515449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.518096+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:25:19.518096+0000","last_peered":"2024-02-26T06:25:19.518096+0000","last_clean":"2024-02-26T06:25:19.518096+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:25:19.518096+0000","last_undegraded":"2024-02-26T06:25:19.518096+0000","last_fullsized":"2024-02-26T06:25:19.518096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181389+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:25:19.181389+0000","last_peered":"2024-02-26T06:25:19.181389+0000","last_clean":"2024-02-26T06:25:19.181389+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:25:19.181389+0000","last_undegraded":"2024-02-26T06:25:19.181389+0000","last_fullsized":"2024-02-26T06:25:19.181389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.213986+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:25:19.213986+0000","last_peered":"2024-02-26T06:25:19.213986+0000","last_clean":"2024-02-26T06:25:19.213986+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:25:19.213986+0000","last_undegraded":"2024-02-26T06:25:19.213986+0000","last_fullsized":"2024-02-26T06:25:19.213986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515800+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:25:19.515800+0000","last_peered":"2024-02-26T06:25:19.515800+0000","last_clean":"2024-02-26T06:25:19.515800+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:25:19.515800+0000","last_undegraded":"2024-02-26T06:25:19.515800+0000","last_fullsized":"2024-02-26T06:25:19.515800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27708,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":208,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":78541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.749},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70599999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.78100000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.84099999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.874},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76400000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1039194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0349999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.35499999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.624},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.069},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57799999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.86599999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83399999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":439429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.4330000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.96899999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.796,"15min":1.796},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.603},{"interface":"front","average":{"1min":1.889,"5min":1.889,"15min":1.889},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":1.6419999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.3080000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"min":{"1min":1.208,"5min":1.208,"15min":1.208},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":18.216000000000001,"5min":18.216000000000001,"15min":18.216000000000001},"last":2.0419999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"min":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.9790000000000001},{"interface":"front","average":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.9239999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":1.2},{"interface":"front","average":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.7529999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"min":{"1min":1.335,"5min":1.335,"15min":1.335},"max":{"1min":3.0270000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.0739999999999998},{"interface":"front","average":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":2.1480000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.8620000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.79100000000000004,"15min":0.80100000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.37,"15min":0.37},"max":{"1min":0.71299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.307,"15min":0.307},"max":{"1min":0.82499999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.441,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.61899999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.436,"15min":0.435},"min":{"1min":0.31,"5min":0.224,"15min":0.224},"max":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41599999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.135,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.505}]},{"osd":4,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.745},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63700000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68999999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58799999999999997,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.357,"5min":1.373,"15min":1.373},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67100000000000004,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84799999999999998,"5min":1.048,"15min":1.048},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.73099999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.874},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.93400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1032422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.309},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.29999999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.51900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.502,"15min":0.502},"max":{"1min":0.97899999999999998,"5min":1.056,"15min":1.056},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.67100000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.497,"15min":0.49399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56699999999999995}]},{"osd":5,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88700000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46300000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85999999999999999,"15min":0.85099999999999998},"min":{"1min":0.65700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85899999999999999,"15min":0.85299999999999998},"min":{"1min":0.59299999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.754}]},{"osd":8,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.53600000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61399999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.54600000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.71099999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":2.0600000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.157,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.5189999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.028},"min":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.802,"5min":1.867,"15min":1.867},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.1299999999999999,"15min":1.1100000000000001},"min":{"1min":0.755,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5940000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.0529999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.1879999999999999,"15min":1.1419999999999999},"min":{"1min":0.99399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7749999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.274,"15min":1.228},"min":{"1min":0.83299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":2.1309999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.276}]},{"osd":3,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.752,"5min":1.5649999999999999,"15min":1.5329999999999999},"min":{"1min":0.93700000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.282,"5min":3.113,"15min":3.113},"last":1.76},{"interface":"front","average":{"1min":1.708,"5min":1.667,"15min":1.661},"min":{"1min":1.113,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.3220000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":2.1930000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3859999999999999,"15min":1.3600000000000001},"min":{"1min":0.93600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.9590000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.748,"5min":1.5620000000000001,"15min":1.51},"min":{"1min":0.91500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.4889999999999999,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.5700000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.6279999999999999,"15min":1.6339999999999999},"min":{"1min":1.0089999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.222,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.371},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.637,"15min":1.6100000000000001},"min":{"1min":0.82699999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.153,"5min":2.798,"15min":2.798},"last":1.9570000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.607,"15min":1.526},"min":{"1min":1.2370000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.673,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":2.0150000000000001},{"interface":"front","average":{"1min":2.0910000000000002,"5min":1.6870000000000001,"15min":1.619},"min":{"1min":1.1910000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7669999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":1.8680000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":1.7649999999999999,"15min":1.7210000000000001},"min":{"1min":1.0720000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.7759999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.9750000000000001,"5min":1.782,"15min":1.75},"min":{"1min":1.347,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.6480000000000001,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":2.3210000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"min":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"max":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":2.4540000000000002},{"interface":"front","average":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"min":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":2.125}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"min":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":2.5179999999999998},{"interface":"front","average":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"min":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":2.5640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.69099999999999995,"15min":0.64500000000000002},"min":{"1min":0.56399999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1910000000000001,"5min":1.252,"15min":1.252},"last":1.048}]},{"osd":2,"last update":"Mon Feb 26 06:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:24:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.314,"5min":0.249,"15min":0.249},"max":{"1min":0.66000000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.33500000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.41699999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91500000000000004},"min":{"1min":0.71599999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.083,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.21}]},{"osd":7,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.60499999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.47099999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.54600000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.88}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":1.087}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":0.88500000000000001,"15min":0.78600000000000003},"min":{"1min":0.51900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.92900000000000005,"15min":0.80300000000000005},"min":{"1min":0.54600000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.806,"5min":3.516,"15min":3.516},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:25:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.96999999999999997,"15min":0.88700000000000001},"min":{"1min":0.72499999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.046},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.1140000000000001,"15min":0.96399999999999997},"min":{"1min":0.95699999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.2430000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:23:38 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.121,"15min":1.042},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":2.028},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.252,"15min":1.1839999999999999},"min":{"1min":0.621,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":2.2480000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:25:01 2024","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.3700000000000001,"15min":1.3200000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":2.1440000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.1559999999999999,"15min":1.139},"min":{"1min":0.63500000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.6719999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.91200000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.325,"15min":1.3089999999999999},"min":{"1min":0.85299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.528},{"interface":"front","average":{"1min":1.8919999999999999,"5min":1.548,"15min":1.502},"min":{"1min":1.3280000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":2.339}]},{"osd":6,"last update":"Mon Feb 26 06:25:01 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.4770000000000001,"15min":1.4350000000000001},"min":{"1min":0.82499999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2999999999999998,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.6499999999999999,"15min":1.5620000000000001},"min":{"1min":1,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.827}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9350000000000001,"5min":1.708,"15min":1.6699999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.7450000000000001},{"interface":"front","average":{"1min":2.0209999999999999,"5min":1.825,"15min":1.7929999999999999},"min":{"1min":1.054,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":2.46}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"min":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":2.0939999999999999},{"interface":"front","average":{"1min":2.323,"5min":2.323,"15min":2.323},"min":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":2.5950000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":2.5350000000000001},{"interface":"front","average":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"min":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.774}]},{"osd":11,"last update":"Mon Feb 26 06:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.875,"15min":1.875},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.95},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.8959999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69599999999999995,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.55800000000000005}]},{"osd":1,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.25,"15min":0.25},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83599999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.437}]},{"osd":3,"last update":"Mon Feb 26 06:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.024,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70399999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.5},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60899999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.872,"15min":0.88},"min":{"1min":0.55000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0489999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.86299999999999999,"15min":0.85099999999999998},"min":{"1min":0.47799999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.143,"5min":1.258,"15min":1.258},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.51900000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.35699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67000000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.123,"5min":1.329,"15min":1.329},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80300000000000005}]},{"osd":10,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.0589999999999999,"15min":0.92800000000000005},"min":{"1min":1.0529999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.155,"15min":0.96499999999999997},"min":{"1min":0.84699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.052,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.97899999999999998,"15min":0.875},"min":{"1min":0.93400000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.0109999999999999,"15min":0.93100000000000005},"min":{"1min":0.88700000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7050000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.1279999999999999,"15min":1.0289999999999999},"min":{"1min":1.05,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.49,"5min":1.1870000000000001,"15min":1.095},"min":{"1min":1.026,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.88500000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:25:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.2430000000000001,"15min":1.0920000000000001},"min":{"1min":1.022,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0590000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.3680000000000001,"15min":1.2290000000000001},"min":{"1min":0.97599999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.323,"5min":3.323,"15min":3.323},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.397,"15min":1.325},"min":{"1min":1.069,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.665,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.8109999999999999,"5min":1.5489999999999999,"15min":1.5069999999999999},"min":{"1min":1.3460000000000001,"5min":0.752,"15min":0.752},"max":{"1min":2.2120000000000002,"5min":5.7770000000000001,"15min":5.7770000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:25:06 2024","interfaces":[{"interface":"back","average":{"1min":2.077,"5min":1.825,"15min":1.738},"min":{"1min":1.3400000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.4359999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.607,"15min":1.478},"min":{"1min":1.095,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.4289999999999998,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":1.2789999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.74,"15min":1.698},"min":{"1min":1.119,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.004},{"interface":"front","average":{"1min":2.121,"5min":1.8819999999999999,"15min":1.8420000000000001},"min":{"1min":1.444,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.5019999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.98199999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"min":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.145},{"interface":"front","average":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"min":{"1min":1.875,"5min":1.875,"15min":1.875},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":1.167}]},{"osd":10,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"min":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.032},{"interface":"front","average":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"min":{"1min":1.677,"5min":1.677,"15min":1.677},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.1890000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.28699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.53100000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.372},"min":{"1min":0.314,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67300000000000004,"5min":1.04,"15min":1.04},"last":0.64000000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74299999999999999,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.442,"15min":0.442},"max":{"1min":0.91600000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.752,"15min":0.70299999999999996},"min":{"1min":0.60899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.94899999999999995}]},{"osd":5,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.41199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.81999999999999995,"15min":0.78800000000000003},"min":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.28,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.64700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.84899999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.33100000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.875,"15min":0.875},"last":0.80100000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.48199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.81999999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.008},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.871}]},{"osd":10,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1031127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.621,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0169999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.94099999999999995,"15min":1.0149999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.073,"5min":7.2249999999999996,"15min":7.2249999999999996},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":1.21,"15min":1.21},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39100000000000001},"min":{"1min":0.29599999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.51300000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.56699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.36699999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.181,"5min":1.298,"15min":1.298},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.751}]},{"osd":4,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.87,"15min":0.87},"last":0.54600000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83799999999999997,"15min":0.85799999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0589999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.82799999999999996,"15min":0.82199999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1200000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":1.196,"15min":1.196},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89600000000000002,"5min":1.177,"15min":1.177},"last":0.61899999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:24:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54200000000000004,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65500000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.47299999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.997}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":23022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:25:27.332 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:25:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 -- 172.21.15.6:0/32953577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c102270 msgr2=0x7fe82c1026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 --2- 172.21.15.6:0/32953577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c102270 0x7fe82c1026f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe818009900 tx=0x7fe81802bc30 comp rx=0 tx=0).stop 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 -- 172.21.15.6:0/32953577 shutdown_connections 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 --2- 172.21.15.6:0/32953577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe82c102c30 0x7fe82c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 --2- 172.21.15.6:0/32953577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c102270 0x7fe82c1026f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 --2- 172.21.15.6:0/32953577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe82c101080 0x7fe82c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 -- 172.21.15.6:0/32953577 >> 172.21.15.6:0/32953577 conn(0x7fe82c0fc610 msgr2=0x7fe82c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 -- 172.21.15.6:0/32953577 shutdown_connections 2024-02-26T06:25:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.597+0000 7fe831e00640 1 -- 172.21.15.6:0/32953577 wait complete. 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 Processor -- start 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 -- start start 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 0x7fe82c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe82c102270 0x7fe82c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe82c102c30 0x7fe82c1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1a0e10 con 0x7fe82c102270 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1a0f80 con 0x7fe82c102c30 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.598+0000 7fe831e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1a1140 con 0x7fe82c101080 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 0x7fe82c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 0x7fe82c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:55590/0 (socket says 172.21.15.6:55590) 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 -- 172.21.15.6:0/1359210926 learned_addr learned my addr 172.21.15.6:0/1359210926 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:27.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe82be00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe82c102270 0x7fe82c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe831400640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe82c102c30 0x7fe82c1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 -- 172.21.15.6:0/1359210926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe82c102c30 msgr2=0x7fe82c1a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe82c102c30 0x7fe82c1a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 -- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe82c102270 msgr2=0x7fe82c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe82c102270 0x7fe82c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 -- 172.21.15.6:0/1359210926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818009560 con 0x7fe82c101080 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.599+0000 7fe830a00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 0x7fe82c199320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe82000e9c0 tx=0x7fe82000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.600+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200096e0 con 0x7fe82c101080 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.600+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe82000cd80 con 0x7fe82c101080 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.600+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820010430 con 0x7fe82c101080 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.600+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c1a1420 con 0x7fe82c101080 2024-02-26T06:25:27.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.600+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c1a18b0 con 0x7fe82c101080 2024-02-26T06:25:27.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.601+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c10cae0 con 0x7fe82c101080 2024-02-26T06:25:27.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.603+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fe820003680 con 0x7fe82c101080 2024-02-26T06:25:27.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.604+0000 7fe829600640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe8140a21c0 0x7fe8140a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:27.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.604+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe820014070 con 0x7fe82c101080 2024-02-26T06:25:27.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.604+0000 7fe82be00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe8140a21c0 0x7fe8140a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:27.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.605+0000 7fe82be00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe8140a21c0 0x7fe8140a4680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe818000c00 tx=0x7fe81802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:27.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.608+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe820096fe0 con 0x7fe82c101080 2024-02-26T06:25:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:27 smithi090 ceph-mon[35681]: pgmap v322: 97 pgs: 97 active+clean; 604 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-02-26T06:25:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:27 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3078853619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:27 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3038161436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:27.615 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: 2024-02-26T06:25:27.445+0000 7fa09026a700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-02-26T06:25:27.615 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:25:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: 2024-02-26T06:25:27.445+0000 7fa09026a700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-02-26T06:25:27.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:27 smithi006 ceph-mon[31790]: pgmap v322: 97 pgs: 97 active+clean; 604 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-02-26T06:25:27.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:27 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3078853619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:27.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:27 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3038161436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:27.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:27 smithi167 ceph-mon[36229]: pgmap v322: 97 pgs: 97 active+clean; 604 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-02-26T06:25:27.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:27 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3078853619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:25:27.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:27 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3038161436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:27.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.803+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe82c074ad0 con 0x7fe8140a21c0 2024-02-26T06:25:27.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.834+0000 7fe829600640 1 -- 172.21.15.6:0/1359210926 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338996 (secure 0 0 0) 0x7fe82c074ad0 con 0x7fe8140a21c0 2024-02-26T06:25:27.837 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:27.839 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe8140a21c0 msgr2=0x7fe8140a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe8140a21c0 0x7fe8140a4680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe818000c00 tx=0x7fe81802c7e0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 msgr2=0x7fe82c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 0x7fe82c199320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe82000e9c0 tx=0x7fe82000ee80 comp rx=0 tx=0).stop 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 shutdown_connections 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe8140a21c0 0x7fe8140a4680 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe82c102c30 0x7fe82c1a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe82c102270 0x7fe82c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 --2- 172.21.15.6:0/1359210926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82c101080 0x7fe82c199320 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:27.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 >> 172.21.15.6:0/1359210926 conn(0x7fe82c0fc610 msgr2=0x7fe82c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:27.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 shutdown_connections 2024-02-26T06:25:27.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:27.842+0000 7fe831e00640 1 -- 172.21.15.6:0/1359210926 wait complete. 2024-02-26T06:25:27.864 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-02-26T06:25:26.108341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618310,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":157,"num_read_kb":136,"num_write":391,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378584,"kb_used_data":12784,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563240,"statfs":{"total":1151940427776,"available":1151552757760,"internally_reserved":0,"allocated":13090816,"data_stored":7096921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27942,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":210,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003633"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516169+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:25:19.516169+0000","last_peered":"2024-02-26T06:25:19.516169+0000","last_clean":"2024-02-26T06:25:19.516169+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:25:19.516169+0000","last_undegraded":"2024-02-26T06:25:19.516169+0000","last_fullsized":"2024-02-26T06:25:19.516169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182322+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:25:19.182322+0000","last_peered":"2024-02-26T06:25:19.182322+0000","last_clean":"2024-02-26T06:25:19.182322+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:25:19.182322+0000","last_undegraded":"2024-02-26T06:25:19.182322+0000","last_fullsized":"2024-02-26T06:25:19.182322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182572+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:25:19.182572+0000","last_peered":"2024-02-26T06:25:19.182572+0000","last_clean":"2024-02-26T06:25:19.182572+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:25:19.182572+0000","last_undegraded":"2024-02-26T06:25:19.182572+0000","last_fullsized":"2024-02-26T06:25:19.182572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516185+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:25:19.516185+0000","last_peered":"2024-02-26T06:25:19.516185+0000","last_clean":"2024-02-26T06:25:19.516185+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:25:19.516185+0000","last_undegraded":"2024-02-26T06:25:19.516185+0000","last_fullsized":"2024-02-26T06:25:19.516185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.371104+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:25:19.371104+0000","last_peered":"2024-02-26T06:25:19.371104+0000","last_clean":"2024-02-26T06:25:19.371104+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:25:19.371104+0000","last_undegraded":"2024-02-26T06:25:19.371104+0000","last_fullsized":"2024-02-26T06:25:19.371104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516246+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:25:19.516246+0000","last_peered":"2024-02-26T06:25:19.516246+0000","last_clean":"2024-02-26T06:25:19.516246+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:25:19.516246+0000","last_undegraded":"2024-02-26T06:25:19.516246+0000","last_fullsized":"2024-02-26T06:25:19.516246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515358+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:25:19.515358+0000","last_peered":"2024-02-26T06:25:19.515358+0000","last_clean":"2024-02-26T06:25:19.515358+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:25:19.515358+0000","last_undegraded":"2024-02-26T06:25:19.515358+0000","last_fullsized":"2024-02-26T06:25:19.515358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.518444+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:25:19.518444+0000","last_peered":"2024-02-26T06:25:19.518444+0000","last_clean":"2024-02-26T06:25:19.518444+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:25:19.518444+0000","last_undegraded":"2024-02-26T06:25:19.518444+0000","last_fullsized":"2024-02-26T06:25:19.518444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181756+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:25:19.181756+0000","last_peered":"2024-02-26T06:25:19.181756+0000","last_clean":"2024-02-26T06:25:19.181756+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:25:19.181756+0000","last_undegraded":"2024-02-26T06:25:19.181756+0000","last_fullsized":"2024-02-26T06:25:19.181756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515786+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:25:19.515786+0000","last_peered":"2024-02-26T06:25:19.515786+0000","last_clean":"2024-02-26T06:25:19.515786+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:25:19.515786+0000","last_undegraded":"2024-02-26T06:25:19.515786+0000","last_fullsized":"2024-02-26T06:25:19.515786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516491+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:25:19.516491+0000","last_peered":"2024-02-26T06:25:19.516491+0000","last_clean":"2024-02-26T06:25:19.516491+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:25:19.516491+0000","last_undegraded":"2024-02-26T06:25:19.516491+0000","last_fullsized":"2024-02-26T06:25:19.516491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516142+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:25:19.516142+0000","last_peered":"2024-02-26T06:25:19.516142+0000","last_clean":"2024-02-26T06:25:19.516142+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:25:19.516142+0000","last_undegraded":"2024-02-26T06:25:19.516142+0000","last_fullsized":"2024-02-26T06:25:19.516142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.213993+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:25:19.213993+0000","last_peered":"2024-02-26T06:25:19.213993+0000","last_clean":"2024-02-26T06:25:19.213993+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:25:19.213993+0000","last_undegraded":"2024-02-26T06:25:19.213993+0000","last_fullsized":"2024-02-26T06:25:19.213993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181531+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:25:19.181531+0000","last_peered":"2024-02-26T06:25:19.181531+0000","last_clean":"2024-02-26T06:25:19.181531+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:25:19.181531+0000","last_undegraded":"2024-02-26T06:25:19.181531+0000","last_fullsized":"2024-02-26T06:25:19.181531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181587+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:25:19.181587+0000","last_peered":"2024-02-26T06:25:19.181587+0000","last_clean":"2024-02-26T06:25:19.181587+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:25:19.181587+0000","last_undegraded":"2024-02-26T06:25:19.181587+0000","last_fullsized":"2024-02-26T06:25:19.181587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181308+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:25:19.181308+0000","last_peered":"2024-02-26T06:25:19.181308+0000","last_clean":"2024-02-26T06:25:19.181308+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:25:19.181308+0000","last_undegraded":"2024-02-26T06:25:19.181308+0000","last_fullsized":"2024-02-26T06:25:19.181308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516298+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:25:19.516298+0000","last_peered":"2024-02-26T06:25:19.516298+0000","last_clean":"2024-02-26T06:25:19.516298+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:25:19.516298+0000","last_undegraded":"2024-02-26T06:25:19.516298+0000","last_fullsized":"2024-02-26T06:25:19.516298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.519897+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:25:19.519897+0000","last_peered":"2024-02-26T06:25:19.519897+0000","last_clean":"2024-02-26T06:25:19.519897+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:25:19.519897+0000","last_undegraded":"2024-02-26T06:25:19.519897+0000","last_fullsized":"2024-02-26T06:25:19.519897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181756+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:25:19.181756+0000","last_peered":"2024-02-26T06:25:19.181756+0000","last_clean":"2024-02-26T06:25:19.181756+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:25:19.181756+0000","last_undegraded":"2024-02-26T06:25:19.181756+0000","last_fullsized":"2024-02-26T06:25:19.181756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.789692+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:25:24.789692+0000","last_peered":"2024-02-26T06:25:24.789692+0000","last_clean":"2024-02-26T06:25:24.789692+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:25:24.789692+0000","last_undegraded":"2024-02-26T06:25:24.789692+0000","last_fullsized":"2024-02-26T06:25:24.789692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520369+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:25:19.520369+0000","last_peered":"2024-02-26T06:25:19.520369+0000","last_clean":"2024-02-26T06:25:19.520369+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:25:19.520369+0000","last_undegraded":"2024-02-26T06:25:19.520369+0000","last_fullsized":"2024-02-26T06:25:19.520369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520788+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:25:19.520788+0000","last_peered":"2024-02-26T06:25:19.520788+0000","last_clean":"2024-02-26T06:25:19.520788+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:25:19.520788+0000","last_undegraded":"2024-02-26T06:25:19.520788+0000","last_fullsized":"2024-02-26T06:25:19.520788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182680+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:25:19.182680+0000","last_peered":"2024-02-26T06:25:19.182680+0000","last_clean":"2024-02-26T06:25:19.182680+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:25:19.182680+0000","last_undegraded":"2024-02-26T06:25:19.182680+0000","last_fullsized":"2024-02-26T06:25:19.182680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'11","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.347149+0000","last_change":"2024-02-26T06:25:17.202246+0000","last_active":"2024-02-26T06:25:24.347149+0000","last_peered":"2024-02-26T06:25:24.347149+0000","last_clean":"2024-02-26T06:25:24.347149+0000","last_became_active":"2024-02-26T06:25:17.201861+0000","last_became_peered":"2024-02-26T06:25:17.201861+0000","last_unstale":"2024-02-26T06:25:24.347149+0000","last_undegraded":"2024-02-26T06:25:24.347149+0000","last_fullsized":"2024-02-26T06:25:24.347149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:14.968655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181365+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:25:19.181365+0000","last_peered":"2024-02-26T06:25:19.181365+0000","last_clean":"2024-02-26T06:25:19.181365+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:25:19.181365+0000","last_undegraded":"2024-02-26T06:25:19.181365+0000","last_fullsized":"2024-02-26T06:25:19.181365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.346132+0000","last_change":"2024-02-26T06:25:17.200321+0000","last_active":"2024-02-26T06:25:24.346132+0000","last_peered":"2024-02-26T06:25:24.346132+0000","last_clean":"2024-02-26T06:25:24.346132+0000","last_became_active":"2024-02-26T06:25:17.199957+0000","last_became_peered":"2024-02-26T06:25:17.199957+0000","last_unstale":"2024-02-26T06:25:24.346132+0000","last_undegraded":"2024-02-26T06:25:24.346132+0000","last_fullsized":"2024-02-26T06:25:24.346132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:20.743399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516570+0000","last_change":"2024-02-26T06:25:18.205619+0000","last_active":"2024-02-26T06:25:19.516570+0000","last_peered":"2024-02-26T06:25:19.516570+0000","last_clean":"2024-02-26T06:25:19.516570+0000","last_became_active":"2024-02-26T06:25:18.205278+0000","last_became_peered":"2024-02-26T06:25:18.205278+0000","last_unstale":"2024-02-26T06:25:19.516570+0000","last_undegraded":"2024-02-26T06:25:19.516570+0000","last_fullsized":"2024-02-26T06:25:19.516570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:48.102668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.287791+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:25:21.287791+0000","last_peered":"2024-02-26T06:25:21.287791+0000","last_clean":"2024-02-26T06:25:21.287791+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:25:21.287791+0000","last_undegraded":"2024-02-26T06:25:21.287791+0000","last_fullsized":"2024-02-26T06:25:21.287791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.371108+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:25:19.371108+0000","last_peered":"2024-02-26T06:25:19.371108+0000","last_clean":"2024-02-26T06:25:19.371108+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:25:19.371108+0000","last_undegraded":"2024-02-26T06:25:19.371108+0000","last_fullsized":"2024-02-26T06:25:19.371108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.317523+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:25:23.317523+0000","last_peered":"2024-02-26T06:25:23.317523+0000","last_clean":"2024-02-26T06:25:23.317523+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:25:23.317523+0000","last_undegraded":"2024-02-26T06:25:23.317523+0000","last_fullsized":"2024-02-26T06:25:23.317523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516419+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:25:19.516419+0000","last_peered":"2024-02-26T06:25:19.516419+0000","last_clean":"2024-02-26T06:25:19.516419+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:25:19.516419+0000","last_undegraded":"2024-02-26T06:25:19.516419+0000","last_fullsized":"2024-02-26T06:25:19.516419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:22.304433+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:25:22.304433+0000","last_peered":"2024-02-26T06:25:22.304433+0000","last_clean":"2024-02-26T06:25:22.304433+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:25:22.304433+0000","last_undegraded":"2024-02-26T06:25:22.304433+0000","last_fullsized":"2024-02-26T06:25:22.304433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.518368+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:25:19.518368+0000","last_peered":"2024-02-26T06:25:19.518368+0000","last_clean":"2024-02-26T06:25:19.518368+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:25:19.518368+0000","last_undegraded":"2024-02-26T06:25:19.518368+0000","last_fullsized":"2024-02-26T06:25:19.518368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.305225+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:25:23.305225+0000","last_peered":"2024-02-26T06:25:23.305225+0000","last_clean":"2024-02-26T06:25:23.305225+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:25:23.305225+0000","last_undegraded":"2024-02-26T06:25:23.305225+0000","last_fullsized":"2024-02-26T06:25:23.305225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.519026+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:25:19.519026+0000","last_peered":"2024-02-26T06:25:19.519026+0000","last_clean":"2024-02-26T06:25:19.519026+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:25:19.519026+0000","last_undegraded":"2024-02-26T06:25:19.519026+0000","last_fullsized":"2024-02-26T06:25:19.519026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.287226+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:25:21.287226+0000","last_peered":"2024-02-26T06:25:21.287226+0000","last_clean":"2024-02-26T06:25:21.287226+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:25:21.287226+0000","last_undegraded":"2024-02-26T06:25:21.287226+0000","last_fullsized":"2024-02-26T06:25:21.287226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181723+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:25:19.181723+0000","last_peered":"2024-02-26T06:25:19.181723+0000","last_clean":"2024-02-26T06:25:19.181723+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:25:19.181723+0000","last_undegraded":"2024-02-26T06:25:19.181723+0000","last_fullsized":"2024-02-26T06:25:19.181723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.345584+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:25:24.345584+0000","last_peered":"2024-02-26T06:25:24.345584+0000","last_clean":"2024-02-26T06:25:24.345584+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:25:24.345584+0000","last_undegraded":"2024-02-26T06:25:24.345584+0000","last_fullsized":"2024-02-26T06:25:24.345584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182851+0000","last_change":"2024-02-26T06:25:18.227237+0000","last_active":"2024-02-26T06:25:19.182851+0000","last_peered":"2024-02-26T06:25:19.182851+0000","last_clean":"2024-02-26T06:25:19.182851+0000","last_became_active":"2024-02-26T06:25:18.226821+0000","last_became_peered":"2024-02-26T06:25:18.226821+0000","last_unstale":"2024-02-26T06:25:19.182851+0000","last_undegraded":"2024-02-26T06:25:19.182851+0000","last_fullsized":"2024-02-26T06:25:19.182851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:23.200468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.287695+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:25:21.287695+0000","last_peered":"2024-02-26T06:25:21.287695+0000","last_clean":"2024-02-26T06:25:21.287695+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:25:21.287695+0000","last_undegraded":"2024-02-26T06:25:21.287695+0000","last_fullsized":"2024-02-26T06:25:21.287695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181545+0000","last_change":"2024-02-26T06:25:18.202144+0000","last_active":"2024-02-26T06:25:19.181545+0000","last_peered":"2024-02-26T06:25:19.181545+0000","last_clean":"2024-02-26T06:25:19.181545+0000","last_became_active":"2024-02-26T06:25:18.201824+0000","last_became_peered":"2024-02-26T06:25:18.201824+0000","last_unstale":"2024-02-26T06:25:19.181545+0000","last_undegraded":"2024-02-26T06:25:19.181545+0000","last_fullsized":"2024-02-26T06:25:19.181545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:31.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'17","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.346374+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:25:24.346374+0000","last_peered":"2024-02-26T06:25:24.346374+0000","last_clean":"2024-02-26T06:25:24.346374+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:25:24.346374+0000","last_undegraded":"2024-02-26T06:25:24.346374+0000","last_fullsized":"2024-02-26T06:25:24.346374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.371194+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:25:19.371194+0000","last_peered":"2024-02-26T06:25:19.371194+0000","last_clean":"2024-02-26T06:25:19.371194+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:25:19.371194+0000","last_undegraded":"2024-02-26T06:25:19.371194+0000","last_fullsized":"2024-02-26T06:25:19.371194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.787867+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:25:24.787867+0000","last_peered":"2024-02-26T06:25:24.787867+0000","last_clean":"2024-02-26T06:25:24.787867+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:25:24.787867+0000","last_undegraded":"2024-02-26T06:25:24.787867+0000","last_fullsized":"2024-02-26T06:25:24.787867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.214388+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:25:19.214388+0000","last_peered":"2024-02-26T06:25:19.214388+0000","last_clean":"2024-02-26T06:25:19.214388+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:25:19.214388+0000","last_undegraded":"2024-02-26T06:25:19.214388+0000","last_fullsized":"2024-02-26T06:25:19.214388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.306753+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:25:23.306753+0000","last_peered":"2024-02-26T06:25:23.306753+0000","last_clean":"2024-02-26T06:25:23.306753+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:25:23.306753+0000","last_undegraded":"2024-02-26T06:25:23.306753+0000","last_fullsized":"2024-02-26T06:25:23.306753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516655+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:25:19.516655+0000","last_peered":"2024-02-26T06:25:19.516655+0000","last_clean":"2024-02-26T06:25:19.516655+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:25:19.516655+0000","last_undegraded":"2024-02-26T06:25:19.516655+0000","last_fullsized":"2024-02-26T06:25:19.516655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.790323+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:25:24.790323+0000","last_peered":"2024-02-26T06:25:24.790323+0000","last_clean":"2024-02-26T06:25:24.790323+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:25:24.790323+0000","last_undegraded":"2024-02-26T06:25:24.790323+0000","last_fullsized":"2024-02-26T06:25:24.790323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516046+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:25:19.516046+0000","last_peered":"2024-02-26T06:25:19.516046+0000","last_clean":"2024-02-26T06:25:19.516046+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:25:19.516046+0000","last_undegraded":"2024-02-26T06:25:19.516046+0000","last_fullsized":"2024-02-26T06:25:19.516046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.314763+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:25:23.314763+0000","last_peered":"2024-02-26T06:25:23.314763+0000","last_clean":"2024-02-26T06:25:23.314763+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:25:23.314763+0000","last_undegraded":"2024-02-26T06:25:23.314763+0000","last_fullsized":"2024-02-26T06:25:23.314763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515611+0000","last_change":"2024-02-26T06:25:18.204602+0000","last_active":"2024-02-26T06:25:19.515611+0000","last_peered":"2024-02-26T06:25:19.515611+0000","last_clean":"2024-02-26T06:25:19.515611+0000","last_became_active":"2024-02-26T06:25:18.204258+0000","last_became_peered":"2024-02-26T06:25:18.204258+0000","last_unstale":"2024-02-26T06:25:19.515611+0000","last_undegraded":"2024-02-26T06:25:19.515611+0000","last_fullsized":"2024-02-26T06:25:19.515611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:59:45.631076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.749660+0000","last_change":"2024-02-26T06:25:17.200714+0000","last_active":"2024-02-26T06:25:19.749660+0000","last_peered":"2024-02-26T06:25:19.749660+0000","last_clean":"2024-02-26T06:25:19.749660+0000","last_became_active":"2024-02-26T06:25:17.200367+0000","last_became_peered":"2024-02-26T06:25:17.200367+0000","last_unstale":"2024-02-26T06:25:19.749660+0000","last_undegraded":"2024-02-26T06:25:19.749660+0000","last_fullsized":"2024-02-26T06:25:19.749660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:15.725139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516187+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:25:19.516187+0000","last_peered":"2024-02-26T06:25:19.516187+0000","last_clean":"2024-02-26T06:25:19.516187+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:25:19.516187+0000","last_undegraded":"2024-02-26T06:25:19.516187+0000","last_fullsized":"2024-02-26T06:25:19.516187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.992901+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:25:19.992901+0000","last_peered":"2024-02-26T06:25:19.992901+0000","last_clean":"2024-02-26T06:25:19.992901+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:25:19.992901+0000","last_undegraded":"2024-02-26T06:25:19.992901+0000","last_fullsized":"2024-02-26T06:25:19.992901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515868+0000","last_change":"2024-02-26T06:25:18.212398+0000","last_active":"2024-02-26T06:25:19.515868+0000","last_peered":"2024-02-26T06:25:19.515868+0000","last_clean":"2024-02-26T06:25:19.515868+0000","last_became_active":"2024-02-26T06:25:18.212021+0000","last_became_peered":"2024-02-26T06:25:18.212021+0000","last_unstale":"2024-02-26T06:25:19.515868+0000","last_undegraded":"2024-02-26T06:25:19.515868+0000","last_fullsized":"2024-02-26T06:25:19.515868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:21.364023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.791911+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:25:24.791911+0000","last_peered":"2024-02-26T06:25:24.791911+0000","last_clean":"2024-02-26T06:25:24.791911+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:25:24.791911+0000","last_undegraded":"2024-02-26T06:25:24.791911+0000","last_fullsized":"2024-02-26T06:25:24.791911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516899+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:25:19.516899+0000","last_peered":"2024-02-26T06:25:19.516899+0000","last_clean":"2024-02-26T06:25:19.516899+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:25:19.516899+0000","last_undegraded":"2024-02-26T06:25:19.516899+0000","last_fullsized":"2024-02-26T06:25:19.516899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515694+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:25:19.515694+0000","last_peered":"2024-02-26T06:25:19.515694+0000","last_clean":"2024-02-26T06:25:19.515694+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:25:19.515694+0000","last_undegraded":"2024-02-26T06:25:19.515694+0000","last_fullsized":"2024-02-26T06:25:19.515694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.787855+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:25:24.787855+0000","last_peered":"2024-02-26T06:25:24.787855+0000","last_clean":"2024-02-26T06:25:24.787855+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:25:24.787855+0000","last_undegraded":"2024-02-26T06:25:24.787855+0000","last_fullsized":"2024-02-26T06:25:24.787855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.519792+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:25:19.519792+0000","last_peered":"2024-02-26T06:25:19.519792+0000","last_clean":"2024-02-26T06:25:19.519792+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:25:19.519792+0000","last_undegraded":"2024-02-26T06:25:19.519792+0000","last_fullsized":"2024-02-26T06:25:19.519792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.787693+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:25:24.787693+0000","last_peered":"2024-02-26T06:25:24.787693+0000","last_clean":"2024-02-26T06:25:24.787693+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:25:24.787693+0000","last_undegraded":"2024-02-26T06:25:24.787693+0000","last_fullsized":"2024-02-26T06:25:24.787693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515988+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:25:19.515988+0000","last_peered":"2024-02-26T06:25:19.515988+0000","last_clean":"2024-02-26T06:25:19.515988+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:25:19.515988+0000","last_undegraded":"2024-02-26T06:25:19.515988+0000","last_fullsized":"2024-02-26T06:25:19.515988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.213536+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:25:19.213536+0000","last_peered":"2024-02-26T06:25:19.213536+0000","last_clean":"2024-02-26T06:25:19.213536+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:25:19.213536+0000","last_undegraded":"2024-02-26T06:25:19.213536+0000","last_fullsized":"2024-02-26T06:25:19.213536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:22.295320+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:25:22.295320+0000","last_peered":"2024-02-26T06:25:22.295320+0000","last_clean":"2024-02-26T06:25:22.295320+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:25:22.295320+0000","last_undegraded":"2024-02-26T06:25:22.295320+0000","last_fullsized":"2024-02-26T06:25:22.295320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.314887+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:25:23.314887+0000","last_peered":"2024-02-26T06:25:23.314887+0000","last_clean":"2024-02-26T06:25:23.314887+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:25:23.314887+0000","last_undegraded":"2024-02-26T06:25:23.314887+0000","last_fullsized":"2024-02-26T06:25:23.314887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.214421+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:25:19.214421+0000","last_peered":"2024-02-26T06:25:19.214421+0000","last_clean":"2024-02-26T06:25:19.214421+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:25:19.214421+0000","last_undegraded":"2024-02-26T06:25:19.214421+0000","last_fullsized":"2024-02-26T06:25:19.214421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.317512+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:25:23.317512+0000","last_peered":"2024-02-26T06:25:23.317512+0000","last_clean":"2024-02-26T06:25:23.317512+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:25:23.317512+0000","last_undegraded":"2024-02-26T06:25:23.317512+0000","last_fullsized":"2024-02-26T06:25:23.317512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520757+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:25:19.520757+0000","last_peered":"2024-02-26T06:25:19.520757+0000","last_clean":"2024-02-26T06:25:19.520757+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:25:19.520757+0000","last_undegraded":"2024-02-26T06:25:19.520757+0000","last_fullsized":"2024-02-26T06:25:19.520757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.349057+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:25:24.349057+0000","last_peered":"2024-02-26T06:25:24.349057+0000","last_clean":"2024-02-26T06:25:24.349057+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:25:24.349057+0000","last_undegraded":"2024-02-26T06:25:24.349057+0000","last_fullsized":"2024-02-26T06:25:24.349057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.182424+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:25:19.182424+0000","last_peered":"2024-02-26T06:25:19.182424+0000","last_clean":"2024-02-26T06:25:19.182424+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:25:19.182424+0000","last_undegraded":"2024-02-26T06:25:19.182424+0000","last_fullsized":"2024-02-26T06:25:19.182424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.286690+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:25:21.286690+0000","last_peered":"2024-02-26T06:25:21.286690+0000","last_clean":"2024-02-26T06:25:21.286690+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:25:21.286690+0000","last_undegraded":"2024-02-26T06:25:21.286690+0000","last_fullsized":"2024-02-26T06:25:21.286690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.370895+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:25:19.370895+0000","last_peered":"2024-02-26T06:25:19.370895+0000","last_clean":"2024-02-26T06:25:19.370895+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:25:19.370895+0000","last_undegraded":"2024-02-26T06:25:19.370895+0000","last_fullsized":"2024-02-26T06:25:19.370895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.289515+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:25:21.289515+0000","last_peered":"2024-02-26T06:25:21.289515+0000","last_clean":"2024-02-26T06:25:21.289515+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:25:21.289515+0000","last_undegraded":"2024-02-26T06:25:21.289515+0000","last_fullsized":"2024-02-26T06:25:21.289515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515778+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:25:19.515778+0000","last_peered":"2024-02-26T06:25:19.515778+0000","last_clean":"2024-02-26T06:25:19.515778+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:25:19.515778+0000","last_undegraded":"2024-02-26T06:25:19.515778+0000","last_fullsized":"2024-02-26T06:25:19.515778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.312339+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:25:23.312339+0000","last_peered":"2024-02-26T06:25:23.312339+0000","last_clean":"2024-02-26T06:25:23.312339+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:25:23.312339+0000","last_undegraded":"2024-02-26T06:25:23.312339+0000","last_fullsized":"2024-02-26T06:25:23.312339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520099+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:25:19.520099+0000","last_peered":"2024-02-26T06:25:19.520099+0000","last_clean":"2024-02-26T06:25:19.520099+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:25:19.520099+0000","last_undegraded":"2024-02-26T06:25:19.520099+0000","last_fullsized":"2024-02-26T06:25:19.520099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:21.286487+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:25:21.286487+0000","last_peered":"2024-02-26T06:25:21.286487+0000","last_clean":"2024-02-26T06:25:21.286487+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:25:21.286487+0000","last_undegraded":"2024-02-26T06:25:21.286487+0000","last_fullsized":"2024-02-26T06:25:21.286487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516881+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:25:19.516881+0000","last_peered":"2024-02-26T06:25:19.516881+0000","last_clean":"2024-02-26T06:25:19.516881+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:25:19.516881+0000","last_undegraded":"2024-02-26T06:25:19.516881+0000","last_fullsized":"2024-02-26T06:25:19.516881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.792620+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:25:24.792620+0000","last_peered":"2024-02-26T06:25:24.792620+0000","last_clean":"2024-02-26T06:25:24.792620+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:25:24.792620+0000","last_undegraded":"2024-02-26T06:25:24.792620+0000","last_fullsized":"2024-02-26T06:25:24.792620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516588+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:25:19.516588+0000","last_peered":"2024-02-26T06:25:19.516588+0000","last_clean":"2024-02-26T06:25:19.516588+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:25:19.516588+0000","last_undegraded":"2024-02-26T06:25:19.516588+0000","last_fullsized":"2024-02-26T06:25:19.516588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:23.308763+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:25:23.308763+0000","last_peered":"2024-02-26T06:25:23.308763+0000","last_clean":"2024-02-26T06:25:23.308763+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:25:23.308763+0000","last_undegraded":"2024-02-26T06:25:23.308763+0000","last_fullsized":"2024-02-26T06:25:23.308763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:20.479971+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:25:20.479971+0000","last_peered":"2024-02-26T06:25:20.479971+0000","last_clean":"2024-02-26T06:25:20.479971+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:25:20.479971+0000","last_undegraded":"2024-02-26T06:25:20.479971+0000","last_fullsized":"2024-02-26T06:25:20.479971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:24.348595+0000","last_change":"2024-02-26T06:25:17.217008+0000","last_active":"2024-02-26T06:25:24.348595+0000","last_peered":"2024-02-26T06:25:24.348595+0000","last_clean":"2024-02-26T06:25:24.348595+0000","last_became_active":"2024-02-26T06:25:17.214955+0000","last_became_peered":"2024-02-26T06:25:17.214955+0000","last_unstale":"2024-02-26T06:25:24.348595+0000","last_undegraded":"2024-02-26T06:25:24.348595+0000","last_fullsized":"2024-02-26T06:25:24.348595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:58:50.627132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.516724+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:25:19.516724+0000","last_peered":"2024-02-26T06:25:19.516724+0000","last_clean":"2024-02-26T06:25:19.516724+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:25:19.516724+0000","last_undegraded":"2024-02-26T06:25:19.516724+0000","last_fullsized":"2024-02-26T06:25:19.516724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515914+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:25:19.515914+0000","last_peered":"2024-02-26T06:25:19.515914+0000","last_clean":"2024-02-26T06:25:19.515914+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:25:19.515914+0000","last_undegraded":"2024-02-26T06:25:19.515914+0000","last_fullsized":"2024-02-26T06:25:19.515914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181474+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:25:19.181474+0000","last_peered":"2024-02-26T06:25:19.181474+0000","last_clean":"2024-02-26T06:25:19.181474+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:25:19.181474+0000","last_undegraded":"2024-02-26T06:25:19.181474+0000","last_fullsized":"2024-02-26T06:25:19.181474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181513+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:25:19.181513+0000","last_peered":"2024-02-26T06:25:19.181513+0000","last_clean":"2024-02-26T06:25:19.181513+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:25:19.181513+0000","last_undegraded":"2024-02-26T06:25:19.181513+0000","last_fullsized":"2024-02-26T06:25:19.181513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181971+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:25:19.181971+0000","last_peered":"2024-02-26T06:25:19.181971+0000","last_clean":"2024-02-26T06:25:19.181971+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:25:19.181971+0000","last_undegraded":"2024-02-26T06:25:19.181971+0000","last_fullsized":"2024-02-26T06:25:19.181971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.214402+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:25:19.214402+0000","last_peered":"2024-02-26T06:25:19.214402+0000","last_clean":"2024-02-26T06:25:19.214402+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:25:19.214402+0000","last_undegraded":"2024-02-26T06:25:19.214402+0000","last_fullsized":"2024-02-26T06:25:19.214402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515978+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:25:19.515978+0000","last_peered":"2024-02-26T06:25:19.515978+0000","last_clean":"2024-02-26T06:25:19.515978+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:25:19.515978+0000","last_undegraded":"2024-02-26T06:25:19.515978+0000","last_fullsized":"2024-02-26T06:25:19.515978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181914+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:25:19.181914+0000","last_peered":"2024-02-26T06:25:19.181914+0000","last_clean":"2024-02-26T06:25:19.181914+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:25:19.181914+0000","last_undegraded":"2024-02-26T06:25:19.181914+0000","last_fullsized":"2024-02-26T06:25:19.181914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.520579+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:25:19.520579+0000","last_peered":"2024-02-26T06:25:19.520579+0000","last_clean":"2024-02-26T06:25:19.520579+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:25:19.520579+0000","last_undegraded":"2024-02-26T06:25:19.520579+0000","last_fullsized":"2024-02-26T06:25:19.520579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515449+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:25:19.515449+0000","last_peered":"2024-02-26T06:25:19.515449+0000","last_clean":"2024-02-26T06:25:19.515449+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:25:19.515449+0000","last_undegraded":"2024-02-26T06:25:19.515449+0000","last_fullsized":"2024-02-26T06:25:19.515449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.518096+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:25:19.518096+0000","last_peered":"2024-02-26T06:25:19.518096+0000","last_clean":"2024-02-26T06:25:19.518096+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:25:19.518096+0000","last_undegraded":"2024-02-26T06:25:19.518096+0000","last_fullsized":"2024-02-26T06:25:19.518096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.181389+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:25:19.181389+0000","last_peered":"2024-02-26T06:25:19.181389+0000","last_clean":"2024-02-26T06:25:19.181389+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:25:19.181389+0000","last_undegraded":"2024-02-26T06:25:19.181389+0000","last_fullsized":"2024-02-26T06:25:19.181389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.213986+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:25:19.213986+0000","last_peered":"2024-02-26T06:25:19.213986+0000","last_clean":"2024-02-26T06:25:19.213986+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:25:19.213986+0000","last_undegraded":"2024-02-26T06:25:19.213986+0000","last_fullsized":"2024-02-26T06:25:19.213986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-26T06:25:19.515800+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:25:19.515800+0000","last_peered":"2024-02-26T06:25:19.515800+0000","last_clean":"2024-02-26T06:25:19.515800+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:25:19.515800+0000","last_undegraded":"2024-02-26T06:25:19.515800+0000","last_fullsized":"2024-02-26T06:25:19.515800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27708,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":208,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":78541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.749},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70599999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.78100000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.84099999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.874},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76400000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1039194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0349999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.35499999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.624},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.069},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57799999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.86599999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83399999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":439429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.4330000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.96899999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.796,"15min":1.796},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.603},{"interface":"front","average":{"1min":1.889,"5min":1.889,"15min":1.889},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":1.6419999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.3080000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"min":{"1min":1.208,"5min":1.208,"15min":1.208},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":18.216000000000001,"5min":18.216000000000001,"15min":18.216000000000001},"last":2.0419999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"min":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.9790000000000001},{"interface":"front","average":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.9239999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":1.2},{"interface":"front","average":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.7529999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"min":{"1min":1.335,"5min":1.335,"15min":1.335},"max":{"1min":3.0270000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.0739999999999998},{"interface":"front","average":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":2.1480000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.8620000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.79100000000000004,"15min":0.80100000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.37,"15min":0.37},"max":{"1min":0.71299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.307,"15min":0.307},"max":{"1min":0.82499999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.441,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.61899999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.436,"15min":0.435},"min":{"1min":0.31,"5min":0.224,"15min":0.224},"max":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41599999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.135,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.505}]},{"osd":4,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.745},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63700000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68999999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58799999999999997,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.357,"5min":1.373,"15min":1.373},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67100000000000004,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84799999999999998,"5min":1.048,"15min":1.048},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.73099999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.874},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.93400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1032422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.309},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.29999999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.51900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.502,"15min":0.502},"max":{"1min":0.97899999999999998,"5min":1.056,"15min":1.056},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.67100000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.497,"15min":0.49399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56699999999999995}]},{"osd":5,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88700000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46300000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85999999999999999,"15min":0.85099999999999998},"min":{"1min":0.65700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85899999999999999,"15min":0.85299999999999998},"min":{"1min":0.59299999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.754}]},{"osd":8,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.53600000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61399999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.54600000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.71099999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":2.0600000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.157,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.5189999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.028},"min":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.802,"5min":1.867,"15min":1.867},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.1299999999999999,"15min":1.1100000000000001},"min":{"1min":0.755,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5940000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.0529999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.1879999999999999,"15min":1.1419999999999999},"min":{"1min":0.99399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7749999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.274,"15min":1.228},"min":{"1min":0.83299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":2.1309999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.276}]},{"osd":3,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.752,"5min":1.5649999999999999,"15min":1.5329999999999999},"min":{"1min":0.93700000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.282,"5min":3.113,"15min":3.113},"last":1.76},{"interface":"front","average":{"1min":1.708,"5min":1.667,"15min":1.661},"min":{"1min":1.113,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.3220000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":2.1930000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3859999999999999,"15min":1.3600000000000001},"min":{"1min":0.93600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.9590000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.748,"5min":1.5620000000000001,"15min":1.51},"min":{"1min":0.91500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.4889999999999999,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.5700000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:25:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.6279999999999999,"15min":1.6339999999999999},"min":{"1min":1.0089999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.222,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.371},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.637,"15min":1.6100000000000001},"min":{"1min":0.82699999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.153,"5min":2.798,"15min":2.798},"last":1.9570000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.607,"15min":1.526},"min":{"1min":1.2370000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.673,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":2.0150000000000001},{"interface":"front","average":{"1min":2.0910000000000002,"5min":1.6870000000000001,"15min":1.619},"min":{"1min":1.1910000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7669999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":1.8680000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":1.7649999999999999,"15min":1.7210000000000001},"min":{"1min":1.0720000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.7759999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.9750000000000001,"5min":1.782,"15min":1.75},"min":{"1min":1.347,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.6480000000000001,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":2.3210000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"min":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"max":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":2.4540000000000002},{"interface":"front","average":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"min":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":2.125}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"min":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":2.5179999999999998},{"interface":"front","average":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"min":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":2.5640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.69099999999999995,"15min":0.64500000000000002},"min":{"1min":0.56399999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1910000000000001,"5min":1.252,"15min":1.252},"last":1.048}]},{"osd":2,"last update":"Mon Feb 26 06:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.876,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:24:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.314,"5min":0.249,"15min":0.249},"max":{"1min":0.66000000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.33500000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.41699999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91500000000000004},"min":{"1min":0.71599999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.083,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.21}]},{"osd":7,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.60499999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.47099999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.54600000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.88}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":1.087}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":0.88500000000000001,"15min":0.78600000000000003},"min":{"1min":0.51900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.92900000000000005,"15min":0.80300000000000005},"min":{"1min":0.54600000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.806,"5min":3.516,"15min":3.516},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:25:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.96999999999999997,"15min":0.88700000000000001},"min":{"1min":0.72499999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.046},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.1140000000000001,"15min":0.96399999999999997},"min":{"1min":0.95699999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.2430000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:23:38 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.121,"15min":1.042},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":2.028},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.252,"15min":1.1839999999999999},"min":{"1min":0.621,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":2.2480000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:25:01 2024","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.3700000000000001,"15min":1.3200000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":2.1440000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.1559999999999999,"15min":1.139},"min":{"1min":0.63500000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.6719999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.91200000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.325,"15min":1.3089999999999999},"min":{"1min":0.85299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.528},{"interface":"front","average":{"1min":1.8919999999999999,"5min":1.548,"15min":1.502},"min":{"1min":1.3280000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":2.339}]},{"osd":6,"last update":"Mon Feb 26 06:25:01 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.4770000000000001,"15min":1.4350000000000001},"min":{"1min":0.82499999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2999999999999998,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.6499999999999999,"15min":1.5620000000000001},"min":{"1min":1,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.827}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9350000000000001,"5min":1.708,"15min":1.6699999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.7450000000000001},{"interface":"front","average":{"1min":2.0209999999999999,"5min":1.825,"15min":1.7929999999999999},"min":{"1min":1.054,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":2.46}]},{"osd":9,"last update":"Mon Feb 26 06:24:25 2024","interfaces":[{"interface":"back","average":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"min":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":2.0939999999999999},{"interface":"front","average":{"1min":2.323,"5min":2.323,"15min":2.323},"min":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":2.5950000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":2.5350000000000001},{"interface":"front","average":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"min":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.774}]},{"osd":11,"last update":"Mon Feb 26 06:25:18 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.875,"15min":1.875},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.95},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.8959999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69599999999999995,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.55800000000000005}]},{"osd":1,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.25,"15min":0.25},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83599999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.437}]},{"osd":3,"last update":"Mon Feb 26 06:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.024,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70399999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.5},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60899999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.872,"15min":0.88},"min":{"1min":0.55000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0489999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.86299999999999999,"15min":0.85099999999999998},"min":{"1min":0.47799999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.143,"5min":1.258,"15min":1.258},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.51900000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.35699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67000000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:25:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.123,"5min":1.329,"15min":1.329},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80300000000000005}]},{"osd":10,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.0589999999999999,"15min":0.92800000000000005},"min":{"1min":1.0529999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.155,"15min":0.96499999999999997},"min":{"1min":0.84699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.052,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.97899999999999998,"15min":0.875},"min":{"1min":0.93400000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.0109999999999999,"15min":0.93100000000000005},"min":{"1min":0.88700000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7050000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.1279999999999999,"15min":1.0289999999999999},"min":{"1min":1.05,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.49,"5min":1.1870000000000001,"15min":1.095},"min":{"1min":1.026,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.88500000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:25:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.2430000000000001,"15min":1.0920000000000001},"min":{"1min":1.022,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0590000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.3680000000000001,"15min":1.2290000000000001},"min":{"1min":0.97599999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.323,"5min":3.323,"15min":3.323},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.397,"15min":1.325},"min":{"1min":1.069,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.665,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.8109999999999999,"5min":1.5489999999999999,"15min":1.5069999999999999},"min":{"1min":1.3460000000000001,"5min":0.752,"15min":0.752},"max":{"1min":2.2120000000000002,"5min":5.7770000000000001,"15min":5.7770000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:25:06 2024","interfaces":[{"interface":"back","average":{"1min":2.077,"5min":1.825,"15min":1.738},"min":{"1min":1.3400000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.4359999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.607,"15min":1.478},"min":{"1min":1.095,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.4289999999999998,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":1.2789999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.74,"15min":1.698},"min":{"1min":1.119,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.004},{"interface":"front","average":{"1min":2.121,"5min":1.8819999999999999,"15min":1.8420000000000001},"min":{"1min":1.444,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.5019999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.98199999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"min":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.145},{"interface":"front","average":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"min":{"1min":1.875,"5min":1.875,"15min":1.875},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":1.167}]},{"osd":10,"last update":"Mon Feb 26 06:24:52 2024","interfaces":[{"interface":"back","average":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"min":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.032},{"interface":"front","average":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"min":{"1min":1.677,"5min":1.677,"15min":1.677},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.1890000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.28699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.53100000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.372},"min":{"1min":0.314,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67300000000000004,"5min":1.04,"15min":1.04},"last":0.64000000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74299999999999999,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.442,"15min":0.442},"max":{"1min":0.91600000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.752,"15min":0.70299999999999996},"min":{"1min":0.60899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.94899999999999995}]},{"osd":5,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.41199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84399999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.81999999999999995,"15min":0.78800000000000003},"min":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.28,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.64700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.84899999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.33100000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.875,"15min":0.875},"last":0.80100000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.48199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.81999999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.008},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.871}]},{"osd":10,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1031127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.621,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0169999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.94099999999999995,"15min":1.0149999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.073,"5min":7.2249999999999996,"15min":7.2249999999999996},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":1.21,"15min":1.21},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39100000000000001},"min":{"1min":0.29599999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.51300000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.56699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.36699999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.181,"5min":1.298,"15min":1.298},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.751}]},{"osd":4,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.87,"15min":0.87},"last":0.54600000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83799999999999997,"15min":0.85799999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0589999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.82799999999999996,"15min":0.82199999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1200000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":1.196,"15min":1.196},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89600000000000002,"5min":1.177,"15min":1.177},"last":0.61899999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:24:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54200000000000004,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65500000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.47299999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.997}]},{"osd":10,"last update":"Mon Feb 26 06:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":23022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:25:27.870 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T06:25:27.870 INFO:tasks.cephadm:Setting up client nodes... 2024-02-26T06:25:27.871 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-26T06:25:28.380 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:28.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:28 smithi090 ceph-mon[35681]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:25:28.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:28 smithi090 ceph-mon[35681]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:25:28.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:28 smithi006 ceph-mon[31790]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:25:28.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:28 smithi006 ceph-mon[31790]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:25:28.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:28 smithi167 ceph-mon[36229]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:25:28.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:28 smithi167 ceph-mon[36229]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:25:29.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:29 smithi090 ceph-mon[35681]: pgmap v323: 97 pgs: 97 active+clean; 604 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2024-02-26T06:25:29.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:29 smithi006 ceph-mon[31790]: pgmap v323: 97 pgs: 97 active+clean; 604 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2024-02-26T06:25:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:29 smithi167 ceph-mon[36229]: pgmap v323: 97 pgs: 97 active+clean; 604 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2024-02-26T06:25:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 -- 172.21.15.6:0/1272479837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8588107a10 msgr2=0x7f8588109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 --2- 172.21.15.6:0/1272479837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8588107a10 0x7f8588109e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8580009f90 tx=0x7f858000b330 comp rx=0 tx=0).stop 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 -- 172.21.15.6:0/1272479837 shutdown_connections 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 --2- 172.21.15.6:0/1272479837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8588107a10 0x7f8588109e20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 --2- 172.21.15.6:0/1272479837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85881027b0 0x7f8588104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 --2- 172.21.15.6:0/1272479837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 0x7f85881074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 -- 172.21.15.6:0/1272479837 >> 172.21.15.6:0/1272479837 conn(0x7f85880fc160 msgr2=0x7f85880fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.717+0000 7f858fc00700 1 -- 172.21.15.6:0/1272479837 shutdown_connections 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.718+0000 7f858fc00700 1 -- 172.21.15.6:0/1272479837 wait complete. 2024-02-26T06:25:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.719+0000 7f858fc00700 1 Processor -- start 2024-02-26T06:25:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.719+0000 7f858fc00700 1 -- start start 2024-02-26T06:25:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 0x7f8588197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858fc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 0x7f85881981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858fc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8588107a10 0x7f858819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858fc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819f820 con 0x7f8588107a10 2024-02-26T06:25:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819f960 con 0x7f85881027b0 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858fc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819fb20 con 0x7f85881050e0 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858dc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8588107a10 0x7f858819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 0x7f8588197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f8587e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 0x7f85881981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 0x7f8588197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:43468/0 (socket says 172.21.15.6:43468) 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858d200700 1 -- 172.21.15.6:0/3676186472 learned_addr learned my addr 172.21.15.6:0/3676186472 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.720+0000 7f858dc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8588107a10 0x7f858819f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46428/0 (socket says 172.21.15.6:46428) 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858d200700 1 -- 172.21.15.6:0/3676186472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 msgr2=0x7f85881981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858d200700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 0x7f85881981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858d200700 1 -- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8588107a10 msgr2=0x7f858819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858d200700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8588107a10 0x7f858819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858d200700 1 -- 172.21.15.6:0/3676186472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c0095f0 con 0x7f85881027b0 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f8587e00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 0x7f85881981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858d200700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 0x7f8588197ca0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f857c00ec10 tx=0x7f857c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c00cc90 con 0x7f85881027b0 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8580009c40 con 0x7f85881027b0 2024-02-26T06:25:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f857c004500 con 0x7f85881027b0 2024-02-26T06:25:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c010690 con 0x7f85881027b0 2024-02-26T06:25:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.721+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85881a01c0 con 0x7f85881027b0 2024-02-26T06:25:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.723+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85880fdce0 con 0x7f85881027b0 2024-02-26T06:25:30.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.725+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f857c003710 con 0x7f85881027b0 2024-02-26T06:25:30.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.726+0000 7f8585600700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85700a2290 0x7f85700a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:30.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.726+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f857c014070 con 0x7f85881027b0 2024-02-26T06:25:30.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.726+0000 7f8587e00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85700a2290 0x7f85700a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:30.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.727+0000 7f8587e00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85700a2290 0x7f85700a4750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f85881991b0 tx=0x7f8574009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:30.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:30.728+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f857c0970f0 con 0x7f85881027b0 2024-02-26T06:25:31.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.047+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f858810d6e0 con 0x7f85881027b0 2024-02-26T06:25:31.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.057+0000 7f8585600700 1 -- 172.21.15.6:0/3676186472 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f857c09a7b0 con 0x7f85881027b0 2024-02-26T06:25:31.056 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-02-26T06:25:31.056 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDbLtxlqFv7AhAALvs2m6sEbF4wguCtjX8g6Q== 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.059+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85700a2290 msgr2=0x7f85700a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.059+0000 7f858fc00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85700a2290 0x7f85700a4750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f85881991b0 tx=0x7f8574009260 comp rx=0 tx=0).stop 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 msgr2=0x7f8588197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 0x7f8588197ca0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f857c00ec10 tx=0x7f857c00ef20 comp rx=0 tx=0).stop 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 shutdown_connections 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f85700a2290 0x7f85700a4750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8588107a10 0x7f858819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85881027b0 0x7f8588197ca0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 --2- 172.21.15.6:0/3676186472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85881050e0 0x7f85881981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 >> 172.21.15.6:0/3676186472 conn(0x7f85880fc160 msgr2=0x7f85881030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 shutdown_connections 2024-02-26T06:25:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:31.060+0000 7f858fc00700 1 -- 172.21.15.6:0/3676186472 wait complete. 2024-02-26T06:25:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:31 smithi090 ceph-mon[35681]: pgmap v324: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-02-26T06:25:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:31 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3676186472' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:25:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:31 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:25:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:31 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T06:25:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:31 smithi006 ceph-mon[31790]: pgmap v324: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-02-26T06:25:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:31 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3676186472' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:25:31.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:31 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:25:31.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:31 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T06:25:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:31 smithi167 ceph-mon[36229]: pgmap v324: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-02-26T06:25:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:31 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3676186472' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:25:31.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:31 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T06:25:31.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:31 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T06:25:31.963 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:25:31.963 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-26T06:25:31.963 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-26T06:25:32.087 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-26T06:25:32.088 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-26T06:25:32.088 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph mgr dump --format=json 2024-02-26T06:25:32.597 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:33.387 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:33 smithi006 ceph-mon[31790]: pgmap v325: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-02-26T06:25:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:33 smithi090 ceph-mon[35681]: pgmap v325: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-02-26T06:25:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:33 smithi167 ceph-mon[36229]: pgmap v325: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-02-26T06:25:35.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 -- 172.21.15.6:0/2505845842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418069e60 msgr2=0x7fc41806a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 --2- 172.21.15.6:0/2505845842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418069e60 0x7fc41806a2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc404009990 tx=0x7fc404009ca0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 -- 172.21.15.6:0/2505845842 shutdown_connections 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 --2- 172.21.15.6:0/2505845842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418104670 0x7fc418104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 --2- 172.21.15.6:0/2505845842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc418106b60 0x7fc418069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 --2- 172.21.15.6:0/2505845842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418069e60 0x7fc41806a2c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 -- 172.21.15.6:0/2505845842 >> 172.21.15.6:0/2505845842 conn(0x7fc418077aa0 msgr2=0x7fc418077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 -- 172.21.15.6:0/2505845842 shutdown_connections 2024-02-26T06:25:35.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.076+0000 7fc41d600700 1 -- 172.21.15.6:0/2505845842 wait complete. 2024-02-26T06:25:35.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.077+0000 7fc41d600700 1 Processor -- start 2024-02-26T06:25:35.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 -- start start 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 0x7fc4181a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418104670 0x7fc4181a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc418106b60 0x7fc41819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819ede0 con 0x7fc418104670 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819ef50 con 0x7fc418069e60 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc41d600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819f110 con 0x7fc418106b60 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc416a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 0x7fc4181a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc416000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418104670 0x7fc4181a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc416a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 0x7fc4181a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:43482/0 (socket says 172.21.15.6:43482) 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc417400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc418106b60 0x7fc41819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc416a00700 1 -- 172.21.15.6:0/1081979102 learned_addr learned my addr 172.21.15.6:0/1081979102 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:35.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.078+0000 7fc416000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418104670 0x7fc4181a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46436/0 (socket says 172.21.15.6:46436) 2024-02-26T06:25:35.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc416a00700 1 -- 172.21.15.6:0/1081979102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc418106b60 msgr2=0x7fc41819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:35.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc416a00700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc418106b60 0x7fc41819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc416a00700 1 -- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418104670 msgr2=0x7fc4181a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc416a00700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418104670 0x7fc4181a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc416a00700 1 -- 172.21.15.6:0/1081979102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040095f0 con 0x7fc418069e60 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc416a00700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 0x7fc4181a4560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc40800e9a0 tx=0x7fc40800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.079+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408009790 con 0x7fc418069e60 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.080+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc408010430 con 0x7fc418069e60 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.080+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41819f3f0 con 0x7fc418069e60 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.080+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408018680 con 0x7fc418069e60 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.080+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4181ab530 con 0x7fc418069e60 2024-02-26T06:25:35.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.081+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc418101150 con 0x7fc418069e60 2024-02-26T06:25:35.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.083+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fc408010910 con 0x7fc418069e60 2024-02-26T06:25:35.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.084+0000 7fc40f400700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:35.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.084+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc408014070 con 0x7fc418069e60 2024-02-26T06:25:35.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.085+0000 7fc416000700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:35.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.085+0000 7fc416000700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc41819fdf0 tx=0x7fc404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:35.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.086+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc408097290 con 0x7fc418069e60 2024-02-26T06:25:35.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.375+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc418118300 con 0x7fc418069e60 2024-02-26T06:25:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.381+0000 7fc40f400700 1 -- 172.21.15.6:0/1081979102 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261473 (secure 0 0 0) 0x7fc408026090 con 0x7fc418069e60 2024-02-26T06:25:35.385 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:35.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3fc0a2260 msgr2=0x7fc3fc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:35.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc41819fdf0 tx=0x7fc404005fd0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 msgr2=0x7fc4181a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:35.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 0x7fc4181a4560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc40800e9a0 tx=0x7fc40800ecb0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 shutdown_connections 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc418106b60 0x7fc41819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc418069e60 0x7fc4181a4560 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 --2- 172.21.15.6:0/1081979102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc418104670 0x7fc4181a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 >> 172.21.15.6:0/1081979102 conn(0x7fc418077aa0 msgr2=0x7fc418103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.389+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 shutdown_connections 2024-02-26T06:25:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:35.390+0000 7fc41d600700 1 -- 172.21.15.6:0/1081979102 wait complete. 2024-02-26T06:25:35.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:35 smithi090 ceph-mon[35681]: pgmap v326: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-02-26T06:25:35.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:35 smithi006 ceph-mon[31790]: pgmap v326: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-02-26T06:25:35.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:35 smithi167 ceph-mon[36229]: pgmap v326: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-02-26T06:25:36.370 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":324687139},{"type":"v1","addr":"172.21.15.6:6801","nonce":324687139}]},"active_addr":"172.21.15.6:6801/324687139","active_change":"2024-02-26T06:17:11.960032+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.6:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1854217832}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2856776212}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3408505415}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1360414143}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3852458696}]}]} 2024-02-26T06:25:36.376 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-26T06:25:36.376 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-26T06:25:36.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T06:25:36.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:36 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1081979102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T06:25:36.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:36 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1081979102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T06:25:36.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:36 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1081979102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T06:25:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:37.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:37 smithi006 ceph-mon[31790]: pgmap v327: 97 pgs: 97 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 27 op/s 2024-02-26T06:25:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:37 smithi167 ceph-mon[36229]: pgmap v327: 97 pgs: 97 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 27 op/s 2024-02-26T06:25:37.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:37 smithi090 ceph-mon[35681]: pgmap v327: 97 pgs: 97 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 27 op/s 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 -- 172.21.15.6:0/3255905519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c112ad0 msgr2=0x7fd74c114ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 --2- 172.21.15.6:0/3255905519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c112ad0 0x7fd74c114ec0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd74400b0a0 tx=0x7fd74400b3b0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 -- 172.21.15.6:0/3255905519 shutdown_connections 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 --2- 172.21.15.6:0/3255905519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c112ad0 0x7fd74c114ec0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 --2- 172.21.15.6:0/3255905519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd74c0761a0 0x7fd74c076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 --2- 172.21.15.6:0/3255905519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c076ac0 0x7fd74c112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 -- 172.21.15.6:0/3255905519 >> 172.21.15.6:0/3255905519 conn(0x7fd74c0ff940 msgr2=0x7fd74c101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 -- 172.21.15.6:0/3255905519 shutdown_connections 2024-02-26T06:25:39.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.335+0000 7fd751800700 1 -- 172.21.15.6:0/3255905519 wait complete. 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.336+0000 7fd751800700 1 Processor -- start 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 -- start start 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd74c0761a0 0x7fd74c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 0x7fd74c1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c112ad0 0x7fd74c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c19ee90 con 0x7fd74c076ac0 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c19f000 con 0x7fd74c112ad0 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd751800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c19f1c0 con 0x7fd74c0761a0 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd743e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 0x7fd74c1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd743e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 0x7fd74c1a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38150/0 (socket says 172.21.15.6:38150) 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd743e00700 1 -- 172.21.15.6:0/353322606 learned_addr learned my addr 172.21.15.6:0/353322606 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.337+0000 7fd74aa00700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd74c0761a0 0x7fd74c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd74b400700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c112ad0 0x7fd74c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd743e00700 1 -- 172.21.15.6:0/353322606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd74c0761a0 msgr2=0x7fd74c1a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd743e00700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd74c0761a0 0x7fd74c1a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd743e00700 1 -- 172.21.15.6:0/353322606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c112ad0 msgr2=0x7fd74c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd743e00700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c112ad0 0x7fd74c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd743e00700 1 -- 172.21.15.6:0/353322606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd744009d00 con 0x7fd74c076ac0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd74b400700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c112ad0 0x7fd74c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.338+0000 7fd743e00700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 0x7fd74c1a4b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e9a0 tx=0x7fd73400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.339+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734009790 con 0x7fd74c076ac0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.339+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c19f4a0 con 0x7fd74c076ac0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.339+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd734010430 con 0x7fd74c076ac0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.339+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734018680 con 0x7fd74c076ac0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.339+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c1ab640 con 0x7fd74c076ac0 2024-02-26T06:25:39.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.340+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c10dcf0 con 0x7fd74c076ac0 2024-02-26T06:25:39.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.343+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fd734010910 con 0x7fd74c076ac0 2024-02-26T06:25:39.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.345+0000 7fd743400700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd7300a2260 0x7fd7300a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:39.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.345+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd734014070 con 0x7fd74c076ac0 2024-02-26T06:25:39.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.345+0000 7fd74aa00700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd7300a2260 0x7fd7300a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:39.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.346+0000 7fd74aa00700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd7300a2260 0x7fd7300a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009a10 tx=0x7fd73c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:39.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.348+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd734097710 con 0x7fd74c076ac0 2024-02-26T06:25:39.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:39 smithi006 ceph-mon[31790]: pgmap v328: 97 pgs: 97 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-02-26T06:25:39.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.580+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd74c19fcc0 con 0x7fd74c076ac0 2024-02-26T06:25:39.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.583+0000 7fd743400700 1 -- 172.21.15.6:0/353322606 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22114 (secure 0 0 0) 0x7fd7340a3e60 con 0x7fd74c076ac0 2024-02-26T06:25:39.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:39.584 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":85,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:25:19.176219+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:25:39.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.586+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd7300a2260 msgr2=0x7fd7300a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.586+0000 7fd751800700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd7300a2260 0x7fd7300a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009a10 tx=0x7fd73c009190 comp rx=0 tx=0).stop 2024-02-26T06:25:39.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.586+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 msgr2=0x7fd74c1a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:39.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.586+0000 7fd751800700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 0x7fd74c1a4b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e9a0 tx=0x7fd73400ecb0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 shutdown_connections 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd7300a2260 0x7fd7300a4720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd74c112ad0 0x7fd74c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd74c0761a0 0x7fd74c1a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 --2- 172.21.15.6:0/353322606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd74c076ac0 0x7fd74c1a4b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 >> 172.21.15.6:0/353322606 conn(0x7fd74c0ff940 msgr2=0x7fd74c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 shutdown_connections 2024-02-26T06:25:39.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:39.587+0000 7fd751800700 1 -- 172.21.15.6:0/353322606 wait complete. 2024-02-26T06:25:39.667 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:39 smithi090 ceph-mon[35681]: pgmap v328: 97 pgs: 97 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-02-26T06:25:39.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:39 smithi167 ceph-mon[36229]: pgmap v328: 97 pgs: 97 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-02-26T06:25:40.369 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-26T06:25:40.370 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T06:25:40.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:40 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/353322606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:40.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:40 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/353322606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:40.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:40 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/353322606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:41.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:41 smithi006 ceph-mon[31790]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-02-26T06:25:41.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:41 smithi167 ceph-mon[36229]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-02-26T06:25:41.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:41 smithi090 ceph-mon[35681]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-02-26T06:25:43.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 -- 172.21.15.6:0/2574505981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe33410d5c0 msgr2=0x7fe334115190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:43.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 --2- 172.21.15.6:0/2574505981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe33410d5c0 0x7fe334115190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00b0a0 tx=0x7fe32c00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 -- 172.21.15.6:0/2574505981 shutdown_connections 2024-02-26T06:25:43.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 --2- 172.21.15.6:0/2574505981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe33410d5c0 0x7fe334115190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 --2- 172.21.15.6:0/2574505981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3341084e0 0x7fe3341088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 --2- 172.21.15.6:0/2574505981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe334108e90 0x7fe33410cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 -- 172.21.15.6:0/2574505981 >> 172.21.15.6:0/2574505981 conn(0x7fe334075440 msgr2=0x7fe334077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:43.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 -- 172.21.15.6:0/2574505981 shutdown_connections 2024-02-26T06:25:43.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.055+0000 7fe33a600700 1 -- 172.21.15.6:0/2574505981 wait complete. 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.056+0000 7fe33a600700 1 Processor -- start 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.056+0000 7fe33a600700 1 -- start start 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe33a600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 0x7fe3341a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe33a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe334108e90 0x7fe3341a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe33a600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe33410d5c0 0x7fe33419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe33a600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33419ef20 con 0x7fe3341084e0 2024-02-26T06:25:43.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe33a600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33419f090 con 0x7fe334108e90 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe33a600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33419f250 con 0x7fe33410d5c0 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe333400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 0x7fe3341a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe333400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 0x7fe3341a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38178/0 (socket says 172.21.15.6:38178) 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe333400700 1 -- 172.21.15.6:0/2873412610 learned_addr learned my addr 172.21.15.6:0/2873412610 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe333e00700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe33410d5c0 0x7fe33419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.057+0000 7fe332a00700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe334108e90 0x7fe3341a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333e00700 1 -- 172.21.15.6:0/2873412610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe334108e90 msgr2=0x7fe3341a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333e00700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe334108e90 0x7fe3341a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333e00700 1 -- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 msgr2=0x7fe3341a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333e00700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 0x7fe3341a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333e00700 1 -- 172.21.15.6:0/2873412610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c009d00 con 0x7fe33410d5c0 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333400700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 0x7fe3341a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:25:43.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe333e00700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe33410d5c0 0x7fe33419e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00b070 tx=0x7fe32c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00e040 con 0x7fe33410d5c0 2024-02-26T06:25:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33419f4d0 con 0x7fe33410d5c0 2024-02-26T06:25:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe32c01d470 con 0x7fe33410d5c0 2024-02-26T06:25:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.058+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c005260 con 0x7fe33410d5c0 2024-02-26T06:25:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.059+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3341ab5d0 con 0x7fe33410d5c0 2024-02-26T06:25:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.060+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3341108e0 con 0x7fe33410d5c0 2024-02-26T06:25:43.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.062+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fe32c01d5e0 con 0x7fe33410d5c0 2024-02-26T06:25:43.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.063+0000 7fe32be00700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe3180a21e0 0x7fe3180a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:43.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.063+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe32c0cf0a0 con 0x7fe33410d5c0 2024-02-26T06:25:43.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.063+0000 7fe333400700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe3180a21e0 0x7fe3180a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.064+0000 7fe333400700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe3180a21e0 0x7fe3180a46a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe324011550 tx=0x7fe324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.065+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe32c098050 con 0x7fe33410d5c0 2024-02-26T06:25:43.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.293+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3341099c0 con 0x7fe33410d5c0 2024-02-26T06:25:43.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.295+0000 7fe32be00700 1 -- 172.21.15.6:0/2873412610 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22114 (secure 0 0 0) 0x7fe32c09b710 con 0x7fe33410d5c0 2024-02-26T06:25:43.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:25:43.296 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":85,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:25:19.176219+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:25:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe3180a21e0 msgr2=0x7fe3180a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe3180a21e0 0x7fe3180a46a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe324011550 tx=0x7fe324009260 comp rx=0 tx=0).stop 2024-02-26T06:25:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe33410d5c0 msgr2=0x7fe33419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe33410d5c0 0x7fe33419e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00b070 tx=0x7fe32c00bd70 comp rx=0 tx=0).stop 2024-02-26T06:25:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 shutdown_connections 2024-02-26T06:25:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe3180a21e0 0x7fe3180a46a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe33410d5c0 0x7fe33419e770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3341084e0 0x7fe3341a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 --2- 172.21.15.6:0/2873412610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe334108e90 0x7fe3341a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 >> 172.21.15.6:0/2873412610 conn(0x7fe334075440 msgr2=0x7fe334076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 shutdown_connections 2024-02-26T06:25:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:43.298+0000 7fe33a600700 1 -- 172.21.15.6:0/2873412610 wait complete. 2024-02-26T06:25:43.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:43 smithi006 ceph-mon[31790]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-02-26T06:25:43.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:43 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2873412610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:43.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:43 smithi167 ceph-mon[36229]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-02-26T06:25:43.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:43 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2873412610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:43.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:43 smithi090 ceph-mon[35681]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-02-26T06:25:43.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:43 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2873412610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:25:44.229 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-26T06:25:44.230 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-26T06:25:44.230 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-26T06:25:44.230 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-26T06:25:44.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-26T06:25:44.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-26T06:25:44.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-26T06:25:44.232 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-26T06:25:44.232 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-26T06:25:44.233 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-26T06:25:44.233 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-26T06:25:44.233 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-26T06:25:45.339 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.368 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.373 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.374 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.389 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.389 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.389 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.404 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.486 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.501 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:45.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:45 smithi006 ceph-mon[31790]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-02-26T06:25:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:45 smithi167 ceph-mon[36229]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-02-26T06:25:45.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:45 smithi090 ceph-mon[35681]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-02-26T06:25:47.648 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:47 smithi006 ceph-mon[31790]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-02-26T06:25:47.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:47 smithi167 ceph-mon[36229]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-02-26T06:25:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:47 smithi090 ceph-mon[35681]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-02-26T06:25:49.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:49 smithi006 ceph-mon[31790]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 8 op/s 2024-02-26T06:25:49.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:49 smithi167 ceph-mon[36229]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 8 op/s 2024-02-26T06:25:49.820 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:49.820 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:25:49.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:49 smithi090 ceph-mon[35681]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 8 op/s 2024-02-26T06:25:51.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:51 smithi006 ceph-mon[31790]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 852 B/s wr, 11 op/s 2024-02-26T06:25:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:51 smithi167 ceph-mon[36229]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 852 B/s wr, 11 op/s 2024-02-26T06:25:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:51 smithi090 ceph-mon[35681]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 852 B/s wr, 11 op/s 2024-02-26T06:25:53.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:53 smithi006 ceph-mon[31790]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-02-26T06:25:53.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:53 smithi167 ceph-mon[36229]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-02-26T06:25:53.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:53 smithi090 ceph-mon[35681]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-02-26T06:25:55.633 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:55 smithi006 ceph-mon[31790]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-02-26T06:25:55.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:55 smithi167 ceph-mon[36229]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-02-26T06:25:55.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:55 smithi090 ceph-mon[35681]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-02-26T06:25:56.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 -- 172.21.15.6:0/1709189117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 msgr2=0x7fdbd4115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:56.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/1709189117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 0x7fdbd4115060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b0a0 tx=0x7fdbcc00b3b0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 -- 172.21.15.6:0/1709189117 shutdown_connections 2024-02-26T06:25:56.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/1709189117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 0x7fdbd4115060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/1709189117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 0x7fdbd41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/1709189117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd4108db0 0x7fdbd410cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 -- 172.21.15.6:0/1709189117 >> 172.21.15.6:0/1709189117 conn(0x7fdbd4077910 msgr2=0x7fdbd4079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.970+0000 7fdbd3e00700 1 -- 172.21.15.6:0/1709189117 shutdown_connections 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.971+0000 7fdbd3e00700 1 -- 172.21.15.6:0/1709189117 wait complete. 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.971+0000 7fdbd3e00700 1 Processor -- start 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 -- start start 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 0x7fdbd41a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd4108db0 0x7fdbd41a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 0x7fdbd419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419ec50 con 0x7fdbd410d490 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419ed90 con 0x7fdbd4108db0 2024-02-26T06:25:56.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419ef50 con 0x7fdbd4108400 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 0x7fdbd419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbcbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd4108db0 0x7fdbd41a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd2a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 0x7fdbd41a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd2a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 0x7fdbd41a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:52064/0 (socket says 172.21.15.6:52064) 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbcbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd4108db0 0x7fdbd41a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:34632/0 (socket says 172.21.15.6:34632) 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.972+0000 7fdbd2a00700 1 -- 172.21.15.6:0/469429765 learned_addr learned my addr 172.21.15.6:0/469429765 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbd2a00700 1 -- 172.21.15.6:0/469429765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd4108db0 msgr2=0x7fdbd41a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbd2a00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd4108db0 0x7fdbd41a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbd2a00700 1 -- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 msgr2=0x7fdbd419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbd2a00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd410d490 0x7fdbd419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbd2a00700 1 -- 172.21.15.6:0/469429765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc009d00 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbd2a00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 0x7fdbd41a43e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9a0 tx=0x7fdbc400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.973+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4009790 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.974+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbc4010430 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.974+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4018680 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.974+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd419f230 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.974+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd41ab480 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.974+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbd4109890 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.977+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fdbc4010910 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.978+0000 7fdbcb400700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdbb80a21e0 0x7fdbb80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.978+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdbc4014070 con 0x7fdbd4108400 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.978+0000 7fdbcbe00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdbb80a21e0 0x7fdbb80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.978+0000 7fdbcb400700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] conn(0x7fdbb80ab9c0 0x7fdbb80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:56.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.978+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 --> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbb80ae490 con 0x7fdbb80ab9c0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.978+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc40ce920 con 0x7fdbd4108400 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.979+0000 7fdbd3400700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] conn(0x7fdbb80ab9c0 0x7fdbb80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.979+0000 7fdbcbe00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdbb80a21e0 0x7fdbb80a46a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4107560 tx=0x7fdbbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.980+0000 7fdbd3400700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] conn(0x7fdbb80ab9c0 0x7fdbb80adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:54.981+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== osd.2 v2:172.21.15.167:6800/1414245392 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdbb80ae490 con 0x7fdbb80ab9c0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.004+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 --> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbd4118500 con 0x7fdbb80ab9c0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.005+0000 7fdbcb400700 1 -- 172.21.15.6:0/469429765 <== osd.2 v2:172.21.15.167:6800/1414245392 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdbd4118500 con 0x7fdbb80ab9c0 2024-02-26T06:25:56.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.005+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] conn(0x7fdbb80ab9c0 msgr2=0x7fdbb80adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.005+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] conn(0x7fdbb80ab9c0 0x7fdbb80adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdbb80a21e0 msgr2=0x7fdbb80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdbb80a21e0 0x7fdbb80a46a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4107560 tx=0x7fdbbc009260 comp rx=0 tx=0).stop 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 msgr2=0x7fdbd41a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 --2- 172.21.15.6:0/469429765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbd4108400 0x7fdbd41a43e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9a0 tx=0x7fdbc400ecb0 comp rx=0 tx=0).stop 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3400700 1 -- 172.21.15.6:0/469429765 reap_dead start 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 shutdown_connections 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 >> 172.21.15.6:0/469429765 conn(0x7fdbd4077910 msgr2=0x7fdbd4079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 shutdown_connections 2024-02-26T06:25:56.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:55.006+0000 7fdbd3e00700 1 -- 172.21.15.6:0/469429765 wait complete. 2024-02-26T06:25:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 -- 172.21.15.6:0/2062308456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 msgr2=0x7f894c1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 --2- 172.21.15.6:0/2062308456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c1044a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f893c009990 tx=0x7f893c009ca0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 -- 172.21.15.6:0/2062308456 shutdown_connections 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 --2- 172.21.15.6:0/2062308456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f894c1091a0 0x7f894c110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 --2- 172.21.15.6:0/2062308456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c1044a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 --2- 172.21.15.6:0/2062308456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f894c104a70 0x7f894c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 -- 172.21.15.6:0/2062308456 >> 172.21.15.6:0/2062308456 conn(0x7f894c0ff950 msgr2=0x7f894c101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 -- 172.21.15.6:0/2062308456 shutdown_connections 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.387+0000 7f8952800700 1 -- 172.21.15.6:0/2062308456 wait complete. 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.388+0000 7f8952800700 1 Processor -- start 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.388+0000 7f8952800700 1 -- start start 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f8952800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f8952800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f894c104a70 0x7f894c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f8952800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f894c1091a0 0x7f894c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f8952800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c19f290 con 0x7f894c104a70 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f8952800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c19f400 con 0x7f894c1091a0 2024-02-26T06:25:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f8952800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c19f5c0 con 0x7f894c1040c0 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f894b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f894b400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:52090/0 (socket says 172.21.15.6:52090) 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f894b400700 1 -- 172.21.15.6:0/2869327677 learned_addr learned my addr 172.21.15.6:0/2869327677 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f894aa00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f894c104a70 0x7f894c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.389+0000 7f894be00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f894c1091a0 0x7f894c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894b400700 1 -- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f894c1091a0 msgr2=0x7f894c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894b400700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f894c1091a0 0x7f894c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894b400700 1 -- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f894c104a70 msgr2=0x7f894c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894b400700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f894c104a70 0x7f894c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894b400700 1 -- 172.21.15.6:0/2869327677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938009520 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894be00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f894c1091a0 0x7f894c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894aa00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f894c104a70 0x7f894c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f894b400700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c19a3b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f893c000c00 tx=0x7f893c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c01d070 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f893c022470 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c017650 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.390+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c0095f0 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.391+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c1a6f30 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.391+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f894c19fb20 con 0x7f894c1040c0 2024-02-26T06:25:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.393+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f893c0225e0 con 0x7f894c1040c0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.395+0000 7f8943e00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f89300a2260 0x7f89300a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.395+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f893c0cf750 con 0x7f894c1040c0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.395+0000 7f894aa00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f89300a2260 0x7f89300a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.395+0000 7f8943e00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] conn(0x7f89300aba40 0x7f89300ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.395+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 --> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89300ae510 con 0x7f89300aba40 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.395+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f893c0cfac0 con 0x7f894c1040c0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.396+0000 7f894be00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] conn(0x7f89300aba40 0x7f89300ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.396+0000 7f894aa00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f89300a2260 0x7f89300a4720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f894c100d70 tx=0x7f8938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.396+0000 7f894be00700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] conn(0x7f89300aba40 0x7f89300ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.397+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== osd.1 v2:172.21.15.90:6800/2679347906 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f89300ae510 con 0x7f89300aba40 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.416+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 --> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f894c062380 con 0x7f89300aba40 2024-02-26T06:25:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.417+0000 7f8943e00700 1 -- 172.21.15.6:0/2869327677 <== osd.1 v2:172.21.15.90:6800/2679347906 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f894c062380 con 0x7f89300aba40 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.417+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] conn(0x7f89300aba40 msgr2=0x7f89300ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] conn(0x7f89300aba40 0x7f89300ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f89300a2260 msgr2=0x7f89300a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f89300a2260 0x7f89300a4720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f894c100d70 tx=0x7f8938009260 comp rx=0 tx=0).stop 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 msgr2=0x7f894c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 --2- 172.21.15.6:0/2869327677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894c1040c0 0x7f894c19a3b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f893c000c00 tx=0x7f893c005ef0 comp rx=0 tx=0).stop 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f894be00700 1 -- 172.21.15.6:0/2869327677 reap_dead start 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 shutdown_connections 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 >> 172.21.15.6:0/2869327677 conn(0x7f894c0ff950 msgr2=0x7f894c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 shutdown_connections 2024-02-26T06:25:57.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:56.418+0000 7f8952800700 1 -- 172.21.15.6:0/2869327677 wait complete. 2024-02-26T06:25:57.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:57 smithi167 ceph-mon[36229]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-02-26T06:25:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:57 smithi090 ceph-mon[35681]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-02-26T06:25:57.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:57 smithi006 ceph-mon[31790]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-02-26T06:25:59.059 INFO:teuthology.orchestra.run.smithi006.stdout:77309411401 2024-02-26T06:25:59.060 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-26T06:25:59.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:25:59 smithi006 ceph-mon[31790]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-26T06:25:59.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:25:59 smithi167 ceph-mon[36229]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-26T06:25:59.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 -- 172.21.15.6:0/3673900586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 msgr2=0x7f971c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 --2- 172.21.15.6:0/3673900586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 0x7f971c102140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97080099c0 tx=0x7f9708009cd0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 -- 172.21.15.6:0/3673900586 shutdown_connections 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 --2- 172.21.15.6:0/3673900586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971c102680 0x7f971c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 --2- 172.21.15.6:0/3673900586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f971c107ca0 0x7f971c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 --2- 172.21.15.6:0/3673900586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 0x7f971c102140 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 -- 172.21.15.6:0/3673900586 >> 172.21.15.6:0/3673900586 conn(0x7f971c078da0 msgr2=0x7f971c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 -- 172.21.15.6:0/3673900586 shutdown_connections 2024-02-26T06:25:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.850+0000 7f9722200700 1 -- 172.21.15.6:0/3673900586 wait complete. 2024-02-26T06:25:59.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.851+0000 7f9722200700 1 Processor -- start 2024-02-26T06:25:59.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 -- start start 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 0x7f971c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971c102680 0x7f971c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f971c107ca0 0x7f971c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19d040 con 0x7f971c101cc0 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19d1b0 con 0x7f971c107ca0 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f9722200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19d370 con 0x7f971c102680 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f971b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 0x7f971c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f971b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 0x7f971c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52952/0 (socket says 172.21.15.6:52952) 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.852+0000 7f971b400700 1 -- 172.21.15.6:0/1586262554 learned_addr learned my addr 172.21.15.6:0/1586262554 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f971c107ca0 0x7f971c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:59.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971aa00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971c102680 0x7f971c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:59.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 -- 172.21.15.6:0/1586262554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971c102680 msgr2=0x7f971c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:59.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971c102680 0x7f971c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 -- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 msgr2=0x7f971c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:59.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f971c101cc0 0x7f971c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 -- 172.21.15.6:0/1586262554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080095f0 con 0x7f971c107ca0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.853+0000 7f971be00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f971c107ca0 0x7f971c19ca30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f971400efc0 tx=0x7f971400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.854+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714018070 con 0x7f971c107ca0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.854+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f971400f040 con 0x7f971c107ca0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.854+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400cd10 con 0x7f971c107ca0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.854+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c19d5f0 con 0x7f971c107ca0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.854+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c1a4eb0 con 0x7f971c107ca0 2024-02-26T06:25:59.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.855+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f971c111fd0 con 0x7f971c107ca0 2024-02-26T06:25:59.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.858+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f9714004ad0 con 0x7f971c107ca0 2024-02-26T06:25:59.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.859+0000 7f9713e00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f97000a2260 0x7f97000a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:59.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.859+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97140ce200 con 0x7f971c107ca0 2024-02-26T06:25:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.859+0000 7f971b400700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f97000a2260 0x7f97000a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.859+0000 7f9713e00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] conn(0x7f97000aba40 0x7f97000ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:25:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.860+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 --> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97000ae510 con 0x7f97000aba40 2024-02-26T06:25:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.860+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97140ce570 con 0x7f971c107ca0 2024-02-26T06:25:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.860+0000 7f971aa00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] conn(0x7f97000aba40 0x7f97000ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:25:59.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.860+0000 7f971b400700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f97000a2260 0x7f97000a4720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d20 tx=0x7f970c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:59.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.861+0000 7f971aa00700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] conn(0x7f97000aba40 0x7f97000ade60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:25:59.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.861+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== osd.6 v2:172.21.15.6:6818/643219798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f97000ae510 con 0x7f97000aba40 2024-02-26T06:25:59.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:25:59 smithi090 ceph-mon[35681]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-26T06:25:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.898+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 --> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f971c062380 con 0x7f97000aba40 2024-02-26T06:25:59.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.899+0000 7f9713e00700 1 -- 172.21.15.6:0/1586262554 <== osd.6 v2:172.21.15.6:6818/643219798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f971c062380 con 0x7f97000aba40 2024-02-26T06:25:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] conn(0x7f97000aba40 msgr2=0x7f97000ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] conn(0x7f97000aba40 0x7f97000ade60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f97000a2260 msgr2=0x7f97000a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f97000a2260 0x7f97000a4720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d20 tx=0x7f970c009260 comp rx=0 tx=0).stop 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f971c107ca0 msgr2=0x7f971c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 --2- 172.21.15.6:0/1586262554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f971c107ca0 0x7f971c19ca30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f971400efc0 tx=0x7f971400c3a0 comp rx=0 tx=0).stop 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f971be00700 1 -- 172.21.15.6:0/1586262554 reap_dead start 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 shutdown_connections 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 >> 172.21.15.6:0/1586262554 conn(0x7f971c078da0 msgr2=0x7f971c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 shutdown_connections 2024-02-26T06:25:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:25:59.900+0000 7f9722200700 1 -- 172.21.15.6:0/1586262554 wait complete. 2024-02-26T06:26:01.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 -- 172.21.15.6:0/3166564627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde4069920 msgr2=0x7fbde4069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 --2- 172.21.15.6:0/3166564627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde4069920 0x7fbde4069d00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009990 tx=0x7fbdd4009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 -- 172.21.15.6:0/3166564627 shutdown_connections 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 --2- 172.21.15.6:0/3166564627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde410e720 0x7fbde4110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 --2- 172.21.15.6:0/3166564627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde4069920 0x7fbde4069d00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 --2- 172.21.15.6:0/3166564627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde406a240 0x7fbde410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 -- 172.21.15.6:0/3166564627 >> 172.21.15.6:0/3166564627 conn(0x7fbde4077aa0 msgr2=0x7fbde4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.113+0000 7fbde9200700 1 -- 172.21.15.6:0/3166564627 shutdown_connections 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.114+0000 7fbde9200700 1 -- 172.21.15.6:0/3166564627 wait complete. 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.114+0000 7fbde9200700 1 Processor -- start 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 -- start start 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde4069920 0x7fbde419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde406a240 0x7fbde419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde410e720 0x7fbde419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde419f2b0 con 0x7fbde406a240 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde419f420 con 0x7fbde4069920 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde9200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde419f5e0 con 0x7fbde410e720 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde406a240 0x7fbde419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde406a240 0x7fbde419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52974/0 (socket says 172.21.15.6:52974) 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.115+0000 7fbde1400700 1 -- 172.21.15.6:0/361778728 learned_addr learned my addr 172.21.15.6:0/361778728 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1e00700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde4069920 0x7fbde419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde2800700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde410e720 0x7fbde419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1400700 1 -- 172.21.15.6:0/361778728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde410e720 msgr2=0x7fbde419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1400700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde410e720 0x7fbde419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1400700 1 -- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde4069920 msgr2=0x7fbde419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1400700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde4069920 0x7fbde419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1400700 1 -- 172.21.15.6:0/361778728 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40095f0 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbde1400700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde406a240 0x7fbde419a910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e9a0 tx=0x7fbdd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0009790 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdd0010430 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.116+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0018680 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.117+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde419f8c0 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.117+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde41a7120 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.118+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbde4101150 con 0x7fbde406a240 2024-02-26T06:26:01.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.120+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fbdd0010910 con 0x7fbde406a240 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.121+0000 7fbddaa00700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbdc80a2260 0x7fbdc80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.121+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbdd0014070 con 0x7fbde406a240 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.121+0000 7fbde1e00700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbdc80a2260 0x7fbdc80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.122+0000 7fbddaa00700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] conn(0x7fbdc80aba40 0x7fbdc80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.122+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 --> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdc80ae510 con 0x7fbdc80aba40 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.122+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd00ce8b0 con 0x7fbde406a240 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.122+0000 7fbde2800700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] conn(0x7fbdc80aba40 0x7fbdc80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.122+0000 7fbde1e00700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbdc80a2260 0x7fbdc80a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4006010 tx=0x7fbdd400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.123+0000 7fbde2800700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] conn(0x7fbdc80aba40 0x7fbdc80ade60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.123+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== osd.10 v2:172.21.15.90:6824/1676953993 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbdc80ae510 con 0x7fbdc80aba40 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.142+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 --> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbde4114210 con 0x7fbdc80aba40 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.142+0000 7fbddaa00700 1 -- 172.21.15.6:0/361778728 <== osd.10 v2:172.21.15.90:6824/1676953993 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbde4114210 con 0x7fbdc80aba40 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] conn(0x7fbdc80aba40 msgr2=0x7fbdc80ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] conn(0x7fbdc80aba40 0x7fbdc80ade60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbdc80a2260 msgr2=0x7fbdc80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbdc80a2260 0x7fbdc80a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4006010 tx=0x7fbdd400b260 comp rx=0 tx=0).stop 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde406a240 msgr2=0x7fbde419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 --2- 172.21.15.6:0/361778728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbde406a240 0x7fbde419a910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e9a0 tx=0x7fbdd000ecb0 comp rx=0 tx=0).stop 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde2800700 1 -- 172.21.15.6:0/361778728 reap_dead start 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 shutdown_connections 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 >> 172.21.15.6:0/361778728 conn(0x7fbde4077aa0 msgr2=0x7fbde410fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 shutdown_connections 2024-02-26T06:26:01.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:01.143+0000 7fbde9200700 1 -- 172.21.15.6:0/361778728 wait complete. 2024-02-26T06:26:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:01 smithi090 ceph-mon[35681]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-02-26T06:26:02.163 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:01 smithi006 ceph-mon[31790]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-02-26T06:26:02.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:01 smithi167 ceph-mon[36229]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-02-26T06:26:02.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.741+0000 7f8ec3000700 1 -- 172.21.15.6:0/3286604313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 msgr2=0x7f8ebc06a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:02.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.741+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3286604313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 0x7f8ebc06a950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0098f0 tx=0x7f8eac009c00 comp rx=0 tx=0).stop 2024-02-26T06:26:02.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 -- 172.21.15.6:0/3286604313 shutdown_connections 2024-02-26T06:26:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3286604313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ebc106fe0 0x7f8ebc10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3286604313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 0x7f8ebc06a950 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3286604313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ebc06af20 0x7f8ebc1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 -- 172.21.15.6:0/3286604313 >> 172.21.15.6:0/3286604313 conn(0x7f8ebc077ae0 msgr2=0x7f8ebc077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 -- 172.21.15.6:0/3286604313 shutdown_connections 2024-02-26T06:26:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.742+0000 7f8ec3000700 1 -- 172.21.15.6:0/3286604313 wait complete. 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 Processor -- start 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 -- start start 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 0x7f8ebc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ebc06af20 0x7f8ebc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ebc106fe0 0x7f8ebc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19d110 con 0x7f8ebc06a570 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19d280 con 0x7f8ebc06af20 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.743+0000 7f8ec3000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19d440 con 0x7f8ebc106fe0 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 0x7f8ebc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 0x7f8ebc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53002/0 (socket says 172.21.15.6:53002) 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ebc06af20 0x7f8ebc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebbe00700 1 -- 172.21.15.6:0/3784436726 learned_addr learned my addr 172.21.15.6:0/3784436726 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ec1000700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ebc106fe0 0x7f8ebc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:02.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 -- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ebc106fe0 msgr2=0x7f8ebc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:02.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ebc106fe0 0x7f8ebc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 -- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 msgr2=0x7f8ebc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:02.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ebc06a570 0x7f8ebc198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 -- 172.21.15.6:0/3784436726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009520 con 0x7f8ebc06af20 2024-02-26T06:26:02.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.744+0000 7f8ebb400700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ebc06af20 0x7f8ebc198770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc1030e0 tx=0x7f8ea800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:02.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.745+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8010900 con 0x7f8ebc06af20 2024-02-26T06:26:02.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.745+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ea8018420 con 0x7f8ebc06af20 2024-02-26T06:26:02.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.745+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800f680 con 0x7f8ebc06af20 2024-02-26T06:26:02.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.745+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc19d720 con 0x7f8ebc06af20 2024-02-26T06:26:02.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.745+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc1a4f80 con 0x7f8ebc06af20 2024-02-26T06:26:02.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.746+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ebc069bc0 con 0x7f8ebc06af20 2024-02-26T06:26:02.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.748+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f8ea8018590 con 0x7f8ebc06af20 2024-02-26T06:26:02.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.749+0000 7f8eb8c00700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8ea00a21e0 0x7f8ea00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:02.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ea8014070 con 0x7f8ebc06af20 2024-02-26T06:26:02.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8ebbe00700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8ea00a21e0 0x7f8ea00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:02.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8eb8c00700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] conn(0x7f8ea00ab9c0 0x7f8ea00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:02.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 --> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ea00ae490 con 0x7f8ea00ab9c0 2024-02-26T06:26:02.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea80ce3d0 con 0x7f8ebc06af20 2024-02-26T06:26:02.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8ebbe00700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8ea00a21e0 0x7f8ea00a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009fd0 tx=0x7f8eac019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:02.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.750+0000 7f8ec1000700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] conn(0x7f8ea00ab9c0 0x7f8ea00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:02.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.751+0000 7f8ec1000700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] conn(0x7f8ea00ab9c0 0x7f8ea00adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:02.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.752+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== osd.11 v2:172.21.15.167:6824/1697184190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8ea00ae490 con 0x7f8ea00ab9c0 2024-02-26T06:26:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.770+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 --> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ebc112050 con 0x7f8ea00ab9c0 2024-02-26T06:26:02.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.771+0000 7f8eb8c00700 1 -- 172.21.15.6:0/3784436726 <== osd.11 v2:172.21.15.167:6824/1697184190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8ebc112050 con 0x7f8ea00ab9c0 2024-02-26T06:26:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] conn(0x7f8ea00ab9c0 msgr2=0x7f8ea00adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:02.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] conn(0x7f8ea00ab9c0 0x7f8ea00adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8ea00a21e0 msgr2=0x7f8ea00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8ea00a21e0 0x7f8ea00a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009fd0 tx=0x7f8eac019040 comp rx=0 tx=0).stop 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ebc06af20 msgr2=0x7f8ebc198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 --2- 172.21.15.6:0/3784436726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ebc06af20 0x7f8ebc198770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc1030e0 tx=0x7f8ea800eca0 comp rx=0 tx=0).stop 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec1000700 1 -- 172.21.15.6:0/3784436726 reap_dead start 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 shutdown_connections 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 >> 172.21.15.6:0/3784436726 conn(0x7f8ebc077ae0 msgr2=0x7f8ebc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 shutdown_connections 2024-02-26T06:26:02.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:02.772+0000 7f8ec3000700 1 -- 172.21.15.6:0/3784436726 wait complete. 2024-02-26T06:26:03.252 INFO:teuthology.orchestra.run.smithi006.stdout:55834574926 2024-02-26T06:26:03.253 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-26T06:26:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:03 smithi090 ceph-mon[35681]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-26T06:26:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:03 smithi006 ceph-mon[31790]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-26T06:26:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:03 smithi167 ceph-mon[36229]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-26T06:26:04.533 INFO:teuthology.orchestra.run.smithi006.stdout:197568495666 2024-02-26T06:26:04.533 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-26T06:26:05.329 INFO:teuthology.orchestra.run.smithi006.stdout:343597383705 2024-02-26T06:26:05.330 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 -- 172.21.15.6:0/2808927805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 msgr2=0x7f9578102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 --2- 172.21.15.6:0/2808927805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f9578102290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f956c009990 tx=0x7f956c009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 -- 172.21.15.6:0/2808927805 shutdown_connections 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 --2- 172.21.15.6:0/2808927805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9578106f90 0x7f957810eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 --2- 172.21.15.6:0/2808927805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f9578102290 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 --2- 172.21.15.6:0/2808927805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9578102860 0x7f95781068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 -- 172.21.15.6:0/2808927805 >> 172.21.15.6:0/2808927805 conn(0x7f95780fd720 msgr2=0x7f95780ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 -- 172.21.15.6:0/2808927805 shutdown_connections 2024-02-26T06:26:05.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.545+0000 7f9580200700 1 -- 172.21.15.6:0/2808927805 wait complete. 2024-02-26T06:26:05.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.547+0000 7f9580200700 1 Processor -- start 2024-02-26T06:26:05.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.547+0000 7f9580200700 1 -- start start 2024-02-26T06:26:05.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f9580200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f957819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f9580200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9578102860 0x7f957819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f9580200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9578106f90 0x7f957819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f9580200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957819f2b0 con 0x7f9578101eb0 2024-02-26T06:26:05.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f9580200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957819f420 con 0x7f9578106f90 2024-02-26T06:26:05.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f9580200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957819f5e0 con 0x7f9578102860 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f957d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f957819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f957d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f957819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53016/0 (socket says 172.21.15.6:53016) 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f957d800700 1 -- 172.21.15.6:0/2653865818 learned_addr learned my addr 172.21.15.6:0/2653865818 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f957e200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9578106f90 0x7f957819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.548+0000 7f957ce00700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9578102860 0x7f957819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957d800700 1 -- 172.21.15.6:0/2653865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9578102860 msgr2=0x7f957819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957d800700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9578102860 0x7f957819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957d800700 1 -- 172.21.15.6:0/2653865818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9578106f90 msgr2=0x7f957819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957d800700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9578106f90 0x7f957819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957d800700 1 -- 172.21.15.6:0/2653865818 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c0095f0 con 0x7f9578101eb0 2024-02-26T06:26:05.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957e200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9578106f90 0x7f957819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:05.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f957d800700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f957819a3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f956c005910 tx=0x7f956c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c01d070 con 0x7f9578101eb0 2024-02-26T06:26:05.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f956c00b610 con 0x7f9578101eb0 2024-02-26T06:26:05.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957819f860 con 0x7f9578101eb0 2024-02-26T06:26:05.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.549+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0173f0 con 0x7f9578101eb0 2024-02-26T06:26:05.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.550+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95781a7120 con 0x7f9578101eb0 2024-02-26T06:26:05.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.551+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9578112010 con 0x7f9578101eb0 2024-02-26T06:26:05.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.553+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f956c0225e0 con 0x7f9578101eb0 2024-02-26T06:26:05.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.554+0000 7f9576000700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95600a21e0 0x7f95600a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.554+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f956c0cf6f0 con 0x7f9578101eb0 2024-02-26T06:26:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.555+0000 7f957ce00700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95600a21e0 0x7f95600a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.555+0000 7f9576000700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] conn(0x7f95600ab9c0 0x7f95600adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.555+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 --> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95600ae490 con 0x7f95600ab9c0 2024-02-26T06:26:05.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.555+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f956c0cfa60 con 0x7f9578101eb0 2024-02-26T06:26:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.555+0000 7f957e200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] conn(0x7f95600ab9c0 0x7f95600adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.555+0000 7f957ce00700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95600a21e0 0x7f95600a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95780fdde0 tx=0x7f9564009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.556+0000 7f957e200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] conn(0x7f95600ab9c0 0x7f95600adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:05.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.557+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== osd.3 v2:172.21.15.6:6810/3201157546 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f95600ae490 con 0x7f95600ab9c0 2024-02-26T06:26:05.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.575+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 --> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9578075180 con 0x7f95600ab9c0 2024-02-26T06:26:05.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9576000700 1 -- 172.21.15.6:0/2653865818 <== osd.3 v2:172.21.15.6:6810/3201157546 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9578075180 con 0x7f95600ab9c0 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] conn(0x7f95600ab9c0 msgr2=0x7f95600adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9580200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] conn(0x7f95600ab9c0 0x7f95600adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95600a21e0 msgr2=0x7f95600a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9580200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95600a21e0 0x7f95600a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95780fdde0 tx=0x7f9564009260 comp rx=0 tx=0).stop 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 msgr2=0x7f957819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f9580200700 1 --2- 172.21.15.6:0/2653865818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9578101eb0 0x7f957819a3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f956c005910 tx=0x7f956c004730 comp rx=0 tx=0).stop 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.576+0000 7f957e200700 1 -- 172.21.15.6:0/2653865818 reap_dead start 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.577+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 shutdown_connections 2024-02-26T06:26:05.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.577+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 >> 172.21.15.6:0/2653865818 conn(0x7f95780fd720 msgr2=0x7f95780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:05.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.577+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 shutdown_connections 2024-02-26T06:26:05.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:05.577+0000 7f9580200700 1 -- 172.21.15.6:0/2653865818 wait complete. 2024-02-26T06:26:06.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:05 smithi006 ceph-mon[31790]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-26T06:26:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:05 smithi090 ceph-mon[35681]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-26T06:26:06.240 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:05 smithi167 ceph-mon[36229]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-26T06:26:06.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 -- 172.21.15.6:0/3519771996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa390104a90 msgr2=0x7fa390108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:06.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 --2- 172.21.15.6:0/3519771996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa390104a90 0x7fa390108ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa378009990 tx=0x7fa378009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 -- 172.21.15.6:0/3519771996 shutdown_connections 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 --2- 172.21.15.6:0/3519771996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3901091c0 0x7fa390110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 --2- 172.21.15.6:0/3519771996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3901040e0 0x7fa3901044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 --2- 172.21.15.6:0/3519771996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa390104a90 0x7fa390108ae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 -- 172.21.15.6:0/3519771996 >> 172.21.15.6:0/3519771996 conn(0x7fa3900ff970 msgr2=0x7fa390101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 -- 172.21.15.6:0/3519771996 shutdown_connections 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.694+0000 7fa396000700 1 -- 172.21.15.6:0/3519771996 wait complete. 2024-02-26T06:26:06.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.695+0000 7fa396000700 1 Processor -- start 2024-02-26T06:26:06.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.695+0000 7fa396000700 1 -- start start 2024-02-26T06:26:06.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa396000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3901040e0 0x7fa39019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa396000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa390104a90 0x7fa39019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa396000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3901091c0 0x7fa39019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa396000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39019f290 con 0x7fa3901040e0 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa396000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39019f400 con 0x7fa3901091c0 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa396000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39019f5c0 con 0x7fa390104a90 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa38f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3901040e0 0x7fa39019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa38f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3901040e0 0x7fa39019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53024/0 (socket says 172.21.15.6:53024) 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa38f400700 1 -- 172.21.15.6:0/2795947171 learned_addr learned my addr 172.21.15.6:0/2795947171 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa387e00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa390104a90 0x7fa39019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.696+0000 7fa38fe00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3901091c0 0x7fa39019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38f400700 1 -- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa390104a90 msgr2=0x7fa39019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38f400700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa390104a90 0x7fa39019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38f400700 1 -- 172.21.15.6:0/2795947171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3901091c0 msgr2=0x7fa39019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:06.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38f400700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3901091c0 0x7fa39019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38f400700 1 -- 172.21.15.6:0/2795947171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780095f0 con 0x7fa3901040e0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38fe00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3901091c0 0x7fa39019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38f400700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3901040e0 0x7fa39019a3b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa38000d950 tx=0x7fa38000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000f890 con 0x7fa3901040e0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa380010460 con 0x7fa3901040e0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000e610 con 0x7fa3901040e0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.697+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39019f8a0 con 0x7fa3901040e0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.698+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3901a7100 con 0x7fa3901040e0 2024-02-26T06:26:06.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.698+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa390114250 con 0x7fa3901040e0 2024-02-26T06:26:06.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.701+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fa380010960 con 0x7fa3901040e0 2024-02-26T06:26:06.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.702+0000 7fa38cc00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa3740a2260 0x7fa3740a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:06.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.702+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3800cdb00 con 0x7fa3901040e0 2024-02-26T06:26:06.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.702+0000 7fa387e00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa3740a2260 0x7fa3740a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:06.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.702+0000 7fa38cc00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] conn(0x7fa3740aba40 0x7fa3740ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:06.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.702+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 --> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3740ae510 con 0x7fa3740aba40 2024-02-26T06:26:06.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.702+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3800cdfa0 con 0x7fa3901040e0 2024-02-26T06:26:06.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.703+0000 7fa38fe00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] conn(0x7fa3740aba40 0x7fa3740ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:06.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.703+0000 7fa387e00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa3740a2260 0x7fa3740a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa390103070 tx=0x7fa37800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:06.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.703+0000 7fa38fe00700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] conn(0x7fa3740aba40 0x7fa3740ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:06.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.704+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== osd.8 v2:172.21.15.167:6816/734768771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa3740ae510 con 0x7fa3740aba40 2024-02-26T06:26:06.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.722+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 --> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa390062300 con 0x7fa3740aba40 2024-02-26T06:26:06.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.723+0000 7fa38cc00700 1 -- 172.21.15.6:0/2795947171 <== osd.8 v2:172.21.15.167:6816/734768771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa390062300 con 0x7fa3740aba40 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] conn(0x7fa3740aba40 msgr2=0x7fa3740ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] conn(0x7fa3740aba40 0x7fa3740ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa3740a2260 msgr2=0x7fa3740a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa3740a2260 0x7fa3740a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa390103070 tx=0x7fa37800b260 comp rx=0 tx=0).stop 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3901040e0 msgr2=0x7fa39019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 --2- 172.21.15.6:0/2795947171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3901040e0 0x7fa39019a3b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa38000d950 tx=0x7fa38000dc60 comp rx=0 tx=0).stop 2024-02-26T06:26:06.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa38fe00700 1 -- 172.21.15.6:0/2795947171 reap_dead start 2024-02-26T06:26:06.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 shutdown_connections 2024-02-26T06:26:06.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 >> 172.21.15.6:0/2795947171 conn(0x7fa3900ff970 msgr2=0x7fa3901004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:06.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 shutdown_connections 2024-02-26T06:26:06.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:06.724+0000 7fa396000700 1 -- 172.21.15.6:0/2795947171 wait complete. 2024-02-26T06:26:08.102 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:07 smithi006 ceph-mon[31790]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:26:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:07 smithi090 ceph-mon[35681]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:26:08.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:07 smithi167 ceph-mon[36229]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:26:08.440 INFO:teuthology.orchestra.run.smithi006.stdout:309237645340 2024-02-26T06:26:08.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-26T06:26:08.554 INFO:teuthology.orchestra.run.smithi006.stdout:107374182468 2024-02-26T06:26:08.554 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 -- 172.21.15.6:0/1768168463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c069920 msgr2=0x7f317c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 --2- 172.21.15.6:0/1768168463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c069920 0x7f317c069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31680099c0 tx=0x7f3168009cd0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 -- 172.21.15.6:0/1768168463 shutdown_connections 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 --2- 172.21.15.6:0/1768168463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f317c06a2c0 0x7f317c10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 --2- 172.21.15.6:0/1768168463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c107cc0 0x7f317c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 --2- 172.21.15.6:0/1768168463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c069920 0x7f317c069d80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 -- 172.21.15.6:0/1768168463 >> 172.21.15.6:0/1768168463 conn(0x7f317c077aa0 msgr2=0x7f317c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 -- 172.21.15.6:0/1768168463 shutdown_connections 2024-02-26T06:26:08.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.814+0000 7f3181c00700 1 -- 172.21.15.6:0/1768168463 wait complete. 2024-02-26T06:26:08.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.815+0000 7f3181c00700 1 Processor -- start 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.815+0000 7f3181c00700 1 -- start start 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f3181c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c069920 0x7f317c100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f3181c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c06a2c0 0x7f317c101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f3181c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f317c107cc0 0x7f317c1016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f3181c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c1a6560 con 0x7f317c06a2c0 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f3181c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c1a66d0 con 0x7f317c069920 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f3181c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c1a6890 con 0x7f317c107cc0 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f317a000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c06a2c0 0x7f317c101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f317aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c069920 0x7f317c100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f317aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c069920 0x7f317c100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38642/0 (socket says 172.21.15.6:38642) 2024-02-26T06:26:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f317aa00700 1 -- 172.21.15.6:0/2394933744 learned_addr learned my addr 172.21.15.6:0/2394933744 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.816+0000 7f317b400700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f317c107cc0 0x7f317c1016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317aa00700 1 -- 172.21.15.6:0/2394933744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f317c107cc0 msgr2=0x7f317c1016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317aa00700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f317c107cc0 0x7f317c1016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317aa00700 1 -- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c06a2c0 msgr2=0x7f317c101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317aa00700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f317c06a2c0 0x7f317c101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317aa00700 1 -- 172.21.15.6:0/2394933744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680095f0 con 0x7f317c069920 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317b400700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f317c107cc0 0x7f317c1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f317aa00700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c069920 0x7f317c100c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f316c00e9a0 tx=0x7f316c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c009790 con 0x7f317c069920 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317c1a6b70 con 0x7f317c069920 2024-02-26T06:26:08.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316c010430 con 0x7f317c069920 2024-02-26T06:26:08.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.817+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c018680 con 0x7f317c069920 2024-02-26T06:26:08.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.818+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317c1a7090 con 0x7f317c069920 2024-02-26T06:26:08.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.819+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f317c111f00 con 0x7f317c069920 2024-02-26T06:26:08.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.822+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f316c010910 con 0x7f317c069920 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.824+0000 7f3173400700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f31600a2260 0x7f31600a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.824+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f316c014070 con 0x7f317c069920 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.824+0000 7f317a000700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f31600a2260 0x7f31600a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.825+0000 7f3173400700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] conn(0x7f31600aba40 0x7f31600ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.825+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 --> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31600ae510 con 0x7f31600aba40 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.825+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316c0cec80 con 0x7f317c069920 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.825+0000 7f317b400700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] conn(0x7f31600aba40 0x7f31600ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.825+0000 7f317a000700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f31600a2260 0x7f31600a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3168000c00 tx=0x7f3168005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:08.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.826+0000 7f317b400700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] conn(0x7f31600aba40 0x7f31600ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:08.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.827+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== osd.4 v2:172.21.15.90:6808/108774388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f31600ae510 con 0x7f31600aba40 2024-02-26T06:26:08.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.845+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 --> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f317c0623c0 con 0x7f31600aba40 2024-02-26T06:26:08.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3173400700 1 -- 172.21.15.6:0/2394933744 <== osd.4 v2:172.21.15.90:6808/108774388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f317c0623c0 con 0x7f31600aba40 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] conn(0x7f31600aba40 msgr2=0x7f31600ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3181c00700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] conn(0x7f31600aba40 0x7f31600ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f31600a2260 msgr2=0x7f31600a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3181c00700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f31600a2260 0x7f31600a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3168000c00 tx=0x7f3168005fb0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c069920 msgr2=0x7f317c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.846+0000 7f3181c00700 1 --2- 172.21.15.6:0/2394933744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f317c069920 0x7f317c100c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f316c00e9a0 tx=0x7f316c00ecb0 comp rx=0 tx=0).stop 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.847+0000 7f317b400700 1 -- 172.21.15.6:0/2394933744 reap_dead start 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.847+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 shutdown_connections 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.847+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 >> 172.21.15.6:0/2394933744 conn(0x7f317c077aa0 msgr2=0x7f317c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:08.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.847+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 shutdown_connections 2024-02-26T06:26:08.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:08.847+0000 7f3181c00700 1 -- 172.21.15.6:0/2394933744 wait complete. 2024-02-26T06:26:09.518 INFO:teuthology.orchestra.run.smithi006.stdout:253403070505 2024-02-26T06:26:09.519 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-26T06:26:10.087 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:09 smithi006 ceph-mon[31790]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:09 smithi090 ceph-mon[35681]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:10.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.131+0000 7f93f8a00700 1 -- 172.21.15.6:0/1821691813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f410e390 msgr2=0x7f93f4110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.131+0000 7f93f8a00700 1 --2- 172.21.15.6:0/1821691813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f410e390 0x7f93f4110800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009990 tx=0x7f93ec009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 -- 172.21.15.6:0/1821691813 shutdown_connections 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 --2- 172.21.15.6:0/1821691813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f410e390 0x7f93f4110800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 --2- 172.21.15.6:0/1821691813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f40740a0 0x7f93f4074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 --2- 172.21.15.6:0/1821691813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f40749c0 0x7f93f410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 -- 172.21.15.6:0/1821691813 >> 172.21.15.6:0/1821691813 conn(0x7f93f40fd640 msgr2=0x7f93f40ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 -- 172.21.15.6:0/1821691813 shutdown_connections 2024-02-26T06:26:10.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.132+0000 7f93f8a00700 1 -- 172.21.15.6:0/1821691813 wait complete. 2024-02-26T06:26:10.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.133+0000 7f93f8a00700 1 Processor -- start 2024-02-26T06:26:10.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.133+0000 7f93f8a00700 1 -- start start 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f8a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f40740a0 0x7f93f419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f40749c0 0x7f93f419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f8a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f410e390 0x7f93f419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f8a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f180 con 0x7f93f410e390 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f8a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f2f0 con 0x7f93f40749c0 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f8a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f4b0 con 0x7f93f40740a0 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f410e390 0x7f93f419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f410e390 0x7f93f419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60898/0 (socket says 172.21.15.6:60898) 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f3200700 1 -- 172.21.15.6:0/3701069338 learned_addr learned my addr 172.21.15.6:0/3701069338 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93e9e00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f40749c0 0x7f93f419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.134+0000 7f93f2800700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f40740a0 0x7f93f419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93e9e00700 1 -- 172.21.15.6:0/3701069338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f40740a0 msgr2=0x7f93f419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93e9e00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f40740a0 0x7f93f419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93e9e00700 1 -- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f410e390 msgr2=0x7f93f419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93e9e00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f410e390 0x7f93f419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93e9e00700 1 -- 172.21.15.6:0/3701069338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec0095f0 con 0x7f93f40749c0 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93f3200700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93f410e390 0x7f93f419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93e9e00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f40749c0 0x7f93f419a7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9f0 tx=0x7f93dc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:10.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00c9c0 con 0x7f93f40749c0 2024-02-26T06:26:10.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93dc010430 con 0x7f93f40749c0 2024-02-26T06:26:10.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f419f790 con 0x7f93f40749c0 2024-02-26T06:26:10.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.135+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc018680 con 0x7f93f40749c0 2024-02-26T06:26:10.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.136+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41a6ff0 con 0x7f93f40749c0 2024-02-26T06:26:10.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.136+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93f41140c0 con 0x7f93f40749c0 2024-02-26T06:26:10.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.139+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f93dc010910 con 0x7f93f40749c0 2024-02-26T06:26:10.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.140+0000 7f93ebe00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93d80a2260 0x7f93d80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:10.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.140+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93dc014070 con 0x7f93f40749c0 2024-02-26T06:26:10.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.140+0000 7f93f2800700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93d80a2260 0x7f93d80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:10.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.140+0000 7f93ebe00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] conn(0x7f93d80aba40 0x7f93d80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:10.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.140+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 --> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93d80ae510 con 0x7f93d80aba40 2024-02-26T06:26:10.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.140+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93dc0ce7b0 con 0x7f93f40749c0 2024-02-26T06:26:10.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.141+0000 7f93f3200700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] conn(0x7f93d80aba40 0x7f93d80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:10.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.141+0000 7f93f2800700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93d80a2260 0x7f93d80a4720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f540 tx=0x7f93e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:10.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.141+0000 7f93f3200700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] conn(0x7f93d80aba40 0x7f93d80ade60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:10.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.142+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== osd.0 v2:172.21.15.6:6802/2652019447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f93d80ae510 con 0x7f93d80aba40 2024-02-26T06:26:10.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.160+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 --> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93f40ff510 con 0x7f93d80aba40 2024-02-26T06:26:10.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.161+0000 7f93ebe00700 1 -- 172.21.15.6:0/3701069338 <== osd.0 v2:172.21.15.6:6802/2652019447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f93f40ff510 con 0x7f93d80aba40 2024-02-26T06:26:10.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] conn(0x7f93d80aba40 msgr2=0x7f93d80ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:10.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] conn(0x7f93d80aba40 0x7f93d80ade60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:10.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93d80a2260 msgr2=0x7f93d80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:10.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93d80a2260 0x7f93d80a4720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f540 tx=0x7f93e400b040 comp rx=0 tx=0).stop 2024-02-26T06:26:10.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f40749c0 msgr2=0x7f93f419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:10.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 --2- 172.21.15.6:0/3701069338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93f40749c0 0x7f93f419a7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9f0 tx=0x7f93dc00ed00 comp rx=0 tx=0).stop 2024-02-26T06:26:10.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f3200700 1 -- 172.21.15.6:0/3701069338 reap_dead start 2024-02-26T06:26:10.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 shutdown_connections 2024-02-26T06:26:10.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 >> 172.21.15.6:0/3701069338 conn(0x7f93f40fd640 msgr2=0x7f93f41105f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:10.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 shutdown_connections 2024-02-26T06:26:10.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.162+0000 7f93f8a00700 1 -- 172.21.15.6:0/3701069338 wait complete. 2024-02-26T06:26:10.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:09 smithi167 ceph-mon[36229]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:11.091 INFO:teuthology.orchestra.run.smithi006.stdout:34359738454 2024-02-26T06:26:11.091 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-26T06:26:11.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.824+0000 7f15b8000700 1 -- 172.21.15.6:0/3015369142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 msgr2=0x7f15b01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:11.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.824+0000 7f15b8000700 1 --2- 172.21.15.6:0/3015369142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 0x7f15b01080a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b0d0 tx=0x7f15a400b3e0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 -- 172.21.15.6:0/3015369142 shutdown_connections 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 --2- 172.21.15.6:0/3015369142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b006a2c0 0x7f15b010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 --2- 172.21.15.6:0/3015369142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 0x7f15b01080a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 --2- 172.21.15.6:0/3015369142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b0069920 0x7f15b0069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 -- 172.21.15.6:0/3015369142 >> 172.21.15.6:0/3015369142 conn(0x7f15b0077aa0 msgr2=0x7f15b0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 -- 172.21.15.6:0/3015369142 shutdown_connections 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.825+0000 7f15b8000700 1 -- 172.21.15.6:0/3015369142 wait complete. 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.827+0000 7f15b8000700 1 Processor -- start 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.827+0000 7f15b8000700 1 -- start start 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.827+0000 7f15b8000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b0069920 0x7f15b019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.827+0000 7f15b8000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b006a2c0 0x7f15b019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:11.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b8000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 0x7f15b019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b8000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b019f400 con 0x7f15b0107cc0 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b8000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b019f570 con 0x7f15b006a2c0 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b8000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b019f730 con 0x7f15b0069920 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b0069920 0x7f15b019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b006a2c0 0x7f15b019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b0069920 0x7f15b019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:33788/0 (socket says 172.21.15.6:33788) 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b006a2c0 0x7f15b019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38694/0 (socket says 172.21.15.6:38694) 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 -- 172.21.15.6:0/1597279166 learned_addr learned my addr 172.21.15.6:0/1597279166 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b6000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 0x7f15b019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 -- 172.21.15.6:0/1597279166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b006a2c0 msgr2=0x7f15b019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b006a2c0 0x7f15b019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 -- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 msgr2=0x7f15b019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15b0107cc0 0x7f15b019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b5600700 1 -- 172.21.15.6:0/1597279166 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4009d00 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.828+0000 7f15b4c00700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15b006a2c0 0x7f15b019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.829+0000 7f15b5600700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b0069920 0x7f15b019a520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15a400bdc0 tx=0x7f15a400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.829+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a400e040 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.829+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b019f9b0 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.829+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15a401d960 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.829+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a401baf0 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.829+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b01a7270 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.830+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15b0103370 con 0x7f15b0069920 2024-02-26T06:26:11.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.832+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f15a401dad0 con 0x7f15b0069920 2024-02-26T06:26:11.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.833+0000 7f15ae000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f15980a21e0 0x7f15980a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:11.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.833+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f15a40cf3a0 con 0x7f15b0069920 2024-02-26T06:26:11.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.834+0000 7f15b4c00700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f15980a21e0 0x7f15980a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:11.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.834+0000 7f15ae000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] conn(0x7f15980ab9c0 0x7f15980adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:11.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.834+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 --> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15980ae490 con 0x7f15980ab9c0 2024-02-26T06:26:11.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.834+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15a40cf710 con 0x7f15b0069920 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.834+0000 7f15b6000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] conn(0x7f15980ab9c0 0x7f15980adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.834+0000 7f15b4c00700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f15980a21e0 0x7f15980a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b00783c0 tx=0x7f159c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.835+0000 7f15b6000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] conn(0x7f15980ab9c0 0x7f15980adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.836+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== osd.9 v2:172.21.15.6:6826/1514059134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f15980ae490 con 0x7f15980ab9c0 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.854+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 --> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15b0111fe0 con 0x7f15980ab9c0 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.854+0000 7f15ae000700 1 -- 172.21.15.6:0/1597279166 <== osd.9 v2:172.21.15.6:6826/1514059134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f15b0111fe0 con 0x7f15980ab9c0 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] conn(0x7f15980ab9c0 msgr2=0x7f15980adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b8000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] conn(0x7f15980ab9c0 0x7f15980adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f15980a21e0 msgr2=0x7f15980a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b8000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f15980a21e0 0x7f15980a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b00783c0 tx=0x7f159c009260 comp rx=0 tx=0).stop 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b0069920 msgr2=0x7f15b019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:11.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b8000700 1 --2- 172.21.15.6:0/1597279166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b0069920 0x7f15b019a520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15a400bdc0 tx=0x7f15a400bdf0 comp rx=0 tx=0).stop 2024-02-26T06:26:11.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.855+0000 7f15b6000700 1 -- 172.21.15.6:0/1597279166 reap_dead start 2024-02-26T06:26:11.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.856+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 shutdown_connections 2024-02-26T06:26:11.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.856+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 >> 172.21.15.6:0/1597279166 conn(0x7f15b0077aa0 msgr2=0x7f15b010ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:11.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.856+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 shutdown_connections 2024-02-26T06:26:11.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:10.856+0000 7f15b8000700 1 -- 172.21.15.6:0/1597279166 wait complete. 2024-02-26T06:26:12.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:11 smithi090 ceph-mon[35681]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:12.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:11 smithi006 ceph-mon[31790]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:12.196 INFO:teuthology.orchestra.run.smithi006.stdout:287762808867 2024-02-26T06:26:12.197 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-26T06:26:12.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:11 smithi167 ceph-mon[36229]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:12.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 -- 172.21.15.6:0/2834049626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 msgr2=0x7fee3c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:12.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 --2- 172.21.15.6:0/2834049626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 0x7fee3c102290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee300098f0 tx=0x7fee30009c00 comp rx=0 tx=0).stop 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 -- 172.21.15.6:0/2834049626 shutdown_connections 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 --2- 172.21.15.6:0/2834049626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee3c106f90 0x7fee3c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 --2- 172.21.15.6:0/2834049626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 0x7fee3c102290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 --2- 172.21.15.6:0/2834049626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee3c102860 0x7fee3c1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 -- 172.21.15.6:0/2834049626 >> 172.21.15.6:0/2834049626 conn(0x7fee3c0fd720 msgr2=0x7fee3c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 -- 172.21.15.6:0/2834049626 shutdown_connections 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.413+0000 7fee43e00700 1 -- 172.21.15.6:0/2834049626 wait complete. 2024-02-26T06:26:12.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.414+0000 7fee43e00700 1 Processor -- start 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 -- start start 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 0x7fee3c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee3c102860 0x7fee3c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee3c106f90 0x7fee3c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c19f2a0 con 0x7fee3c102860 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c19f410 con 0x7fee3c106f90 2024-02-26T06:26:12.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee43e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c19f5d0 con 0x7fee3c101eb0 2024-02-26T06:26:12.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee40a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee3c102860 0x7fee3c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:12.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee41400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 0x7fee3c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:12.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee40a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee3c102860 0x7fee3c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60928/0 (socket says 172.21.15.6:60928) 2024-02-26T06:26:12.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee40a00700 1 -- 172.21.15.6:0/1081531719 learned_addr learned my addr 172.21.15.6:0/1081531719 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:12.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.415+0000 7fee41e00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee3c106f90 0x7fee3c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:12.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.416+0000 7fee41400700 1 -- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee3c106f90 msgr2=0x7fee3c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:12.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.416+0000 7fee41400700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee3c106f90 0x7fee3c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:12.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.416+0000 7fee41400700 1 -- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee3c102860 msgr2=0x7fee3c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:12.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.416+0000 7fee41400700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee3c102860 0x7fee3c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:12.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.416+0000 7fee41400700 1 -- 172.21.15.6:0/1081531719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee30009520 con 0x7fee3c101eb0 2024-02-26T06:26:12.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.416+0000 7fee41400700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 0x7fee3c19a3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee30000c00 tx=0x7fee3000fa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:12.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.417+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3001c070 con 0x7fee3c101eb0 2024-02-26T06:26:12.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.417+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c19f850 con 0x7fee3c101eb0 2024-02-26T06:26:12.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.417+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee3000b560 con 0x7fee3c101eb0 2024-02-26T06:26:12.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.417+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30003c10 con 0x7fee3c101eb0 2024-02-26T06:26:12.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.417+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c1a7110 con 0x7fee3c101eb0 2024-02-26T06:26:12.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.418+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee3c111fc0 con 0x7fee3c101eb0 2024-02-26T06:26:12.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.420+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fee3000ba60 con 0x7fee3c101eb0 2024-02-26T06:26:12.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.423+0000 7fee3a000700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee240a2260 0x7fee240a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:12.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.423+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee300ce990 con 0x7fee3c101eb0 2024-02-26T06:26:12.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.423+0000 7fee40a00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee240a2260 0x7fee240a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:12.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.423+0000 7fee3a000700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] conn(0x7fee240aba40 0x7fee240ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:12.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.423+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 --> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee240ae510 con 0x7fee240aba40 2024-02-26T06:26:12.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.423+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee300ced00 con 0x7fee3c101eb0 2024-02-26T06:26:12.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.424+0000 7fee40a00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee240a2260 0x7fee240a4720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0feef0 tx=0x7fee28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:12.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.424+0000 7fee41e00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] conn(0x7fee240aba40 0x7fee240ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:12.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.424+0000 7fee41e00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] conn(0x7fee240aba40 0x7fee240ade60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:12.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.425+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== osd.7 v2:172.21.15.90:6816/2240633136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fee240ae510 con 0x7fee240aba40 2024-02-26T06:26:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.444+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 --> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee3c075a10 con 0x7fee240aba40 2024-02-26T06:26:12.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.444+0000 7fee3a000700 1 -- 172.21.15.6:0/1081531719 <== osd.7 v2:172.21.15.90:6816/2240633136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fee3c075a10 con 0x7fee240aba40 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] conn(0x7fee240aba40 msgr2=0x7fee240ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] conn(0x7fee240aba40 0x7fee240ade60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee240a2260 msgr2=0x7fee240a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee240a2260 0x7fee240a4720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0feef0 tx=0x7fee28009260 comp rx=0 tx=0).stop 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 msgr2=0x7fee3c19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 --2- 172.21.15.6:0/1081531719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee3c101eb0 0x7fee3c19a3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee30000c00 tx=0x7fee3000fa90 comp rx=0 tx=0).stop 2024-02-26T06:26:12.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee41e00700 1 -- 172.21.15.6:0/1081531719 reap_dead start 2024-02-26T06:26:12.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 shutdown_connections 2024-02-26T06:26:12.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 >> 172.21.15.6:0/1081531719 conn(0x7fee3c0fd720 msgr2=0x7fee3c10de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:12.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 shutdown_connections 2024-02-26T06:26:12.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:12.445+0000 7fee43e00700 1 -- 172.21.15.6:0/1081531719 wait complete. 2024-02-26T06:26:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-26T06:26:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-02-26T06:26:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-02-26T06:26:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-26T06:26:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-02-26T06:26:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-02-26T06:26:13.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-02-26T06:26:13.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T06:26:13.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:12 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 7]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T06:26:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:12 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 7]}]: dispatch 2024-02-26T06:26:13.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T06:26:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:12 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 7]}]: dispatch 2024-02-26T06:26:13.276 INFO:teuthology.orchestra.run.smithi006.stdout:141733920832 2024-02-26T06:26:13.276 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-26T06:26:13.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 -- 172.21.15.6:0/1703547330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 msgr2=0x7f4b2c074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:13.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1703547330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 0x7f4b2c074420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009990 tx=0x7f4b1c009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 -- 172.21.15.6:0/1703547330 shutdown_connections 2024-02-26T06:26:13.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1703547330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b2c10e530 0x7f4b2c110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1703547330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 0x7f4b2c074420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1703547330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 0x7f4b2c10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 -- 172.21.15.6:0/1703547330 >> 172.21.15.6:0/1703547330 conn(0x7f4b2c0fd740 msgr2=0x7f4b2c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 -- 172.21.15.6:0/1703547330 shutdown_connections 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.828+0000 7f4b30c00700 1 -- 172.21.15.6:0/1703547330 wait complete. 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.829+0000 7f4b30c00700 1 Processor -- start 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 -- start start 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 0x7f4b2c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 0x7f4b2c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b2c10e530 0x7f4b2c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c19f2d0 con 0x7f4b2c074960 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c19f440 con 0x7f4b2c074040 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b30c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c19f600 con 0x7f4b2c10e530 2024-02-26T06:26:13.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b29400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 0x7f4b2c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b29e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 0x7f4b2c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b29400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 0x7f4b2c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60878/0 (socket says 172.21.15.6:60878) 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.830+0000 7f4b29400700 1 -- 172.21.15.6:0/1246966628 learned_addr learned my addr 172.21.15.6:0/1246966628 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b2a800700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b2c10e530 0x7f4b2c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29e00700 1 -- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b2c10e530 msgr2=0x7f4b2c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29e00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b2c10e530 0x7f4b2c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29e00700 1 -- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 msgr2=0x7f4b2c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29e00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 0x7f4b2c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29e00700 1 -- 172.21.15.6:0/1246966628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009520 con 0x7f4b2c074040 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29400700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b2c074960 0x7f4b2c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b29e00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 0x7f4b2c19a3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c000c00 tx=0x7f4b1c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.831+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c01d070 con 0x7f4b2c074040 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.832+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c00b610 con 0x7f4b2c074040 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.832+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c021e20 con 0x7f4b2c074040 2024-02-26T06:26:13.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.832+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c0095f0 con 0x7f4b2c074040 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.832+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c19fb40 con 0x7f4b2c074040 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.833+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b2c1142d0 con 0x7f4b2c074040 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.835+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f4b1c0214a0 con 0x7f4b2c074040 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.836+0000 7f4b22a00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4b100a22a0 0x7f4b100a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.836+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4b1c0ceef0 con 0x7f4b2c074040 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.836+0000 7f4b29400700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4b100a22a0 0x7f4b100a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.837+0000 7f4b22a00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] conn(0x7f4b100aba80 0x7f4b100adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.837+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 --> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b100ae550 con 0x7f4b100aba80 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.837+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b1c0d14d0 con 0x7f4b2c074040 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.837+0000 7f4b2a800700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] conn(0x7f4b100aba80 0x7f4b100adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.837+0000 7f4b29400700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4b100a22a0 0x7f4b100a4760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009fd0 tx=0x7f4b18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.837+0000 7f4b2a800700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] conn(0x7f4b100aba80 0x7f4b100adea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.838+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== osd.5 v2:172.21.15.167:6808/3058964685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4b100ae550 con 0x7f4b100aba80 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.858+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 --> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b2c0623b0 con 0x7f4b100aba80 2024-02-26T06:26:13.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.859+0000 7f4b22a00700 1 -- 172.21.15.6:0/1246966628 <== osd.5 v2:172.21.15.167:6808/3058964685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b2c0623b0 con 0x7f4b100aba80 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] conn(0x7f4b100aba80 msgr2=0x7f4b100adea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] conn(0x7f4b100aba80 0x7f4b100adea0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4b100a22a0 msgr2=0x7f4b100a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4b100a22a0 0x7f4b100a4760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009fd0 tx=0x7f4b18009260 comp rx=0 tx=0).stop 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 msgr2=0x7f4b2c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 --2- 172.21.15.6:0/1246966628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b2c074040 0x7f4b2c19a3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c000c00 tx=0x7f4b1c005740 comp rx=0 tx=0).stop 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b2a800700 1 -- 172.21.15.6:0/1246966628 reap_dead start 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 shutdown_connections 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 >> 172.21.15.6:0/1246966628 conn(0x7f4b2c0fd740 msgr2=0x7f4b2c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 shutdown_connections 2024-02-26T06:26:13.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:09.860+0000 7f4b30c00700 1 -- 172.21.15.6:0/1246966628 wait complete. 2024-02-26T06:26:13.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-26T06:26:13.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-26T06:26:13.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-02-26T06:26:13.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-02-26T06:26:13.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-26T06:26:13.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-02-26T06:26:13.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-02-26T06:26:13.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-02-26T06:26:13.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T06:26:13.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 7]}]': finished 2024-02-26T06:26:13.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:13 smithi006 ceph-mon[31790]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-02-26T06:26:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-02-26T06:26:14.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T06:26:14.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 7]}]': finished 2024-02-26T06:26:14.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:13 smithi090 ceph-mon[35681]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T06:26:14.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 7]}]': finished 2024-02-26T06:26:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:13 smithi167 ceph-mon[36229]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T06:26:14.630 INFO:teuthology.orchestra.run.smithi006.stdout:219043332144 2024-02-26T06:26:14.630 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-26T06:26:14.902 INFO:teuthology.orchestra.run.smithi006.stdout:163208757308 2024-02-26T06:26:14.902 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-26T06:26:14.951 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:15.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:14 smithi090 ceph-mon[35681]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T06:26:15.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:14 smithi006 ceph-mon[31790]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T06:26:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:14 smithi167 ceph-mon[36229]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T06:26:15.959 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:15.965 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:15.965 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:15.968 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:15.968 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:16.002 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:16.018 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:16.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:15 smithi006 ceph-mon[31790]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-02-26T06:26:16.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:15 smithi167 ceph-mon[36229]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-02-26T06:26:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:15 smithi090 ceph-mon[35681]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-02-26T06:26:17.572 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:18.103 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:17 smithi006 ceph-mon[31790]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:18.103 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:17 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:26:18.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:17 smithi167 ceph-mon[36229]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:18.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:17 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:26:18.275 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:17 smithi090 ceph-mon[35681]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:18.275 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:17 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:26:19.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:18 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1310468158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T06:26:19.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.734+0000 7f5332c00700 1 -- 172.21.15.6:0/1412258629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 msgr2=0x7f532c076580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.734+0000 7f5332c00700 1 --2- 172.21.15.6:0/1412258629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c076580 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f531c009990 tx=0x7f531c009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.735+0000 7f5332c00700 1 -- 172.21.15.6:0/1412258629 shutdown_connections 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.735+0000 7f5332c00700 1 --2- 172.21.15.6:0/1412258629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 0x7f532c114f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.735+0000 7f5332c00700 1 --2- 172.21.15.6:0/1412258629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c076580 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.735+0000 7f5332c00700 1 --2- 172.21.15.6:0/1412258629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f532c076ac0 0x7f532c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.735+0000 7f5332c00700 1 -- 172.21.15.6:0/1412258629 >> 172.21.15.6:0/1412258629 conn(0x7f532c0ff970 msgr2=0x7f532c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.736+0000 7f5332c00700 1 -- 172.21.15.6:0/1412258629 shutdown_connections 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.736+0000 7f5332c00700 1 -- 172.21.15.6:0/1412258629 wait complete. 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.738+0000 7f5332c00700 1 Processor -- start 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.738+0000 7f5332c00700 1 -- start start 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5332c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c1a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5332c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f532c076ac0 0x7f532c1a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5332c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 0x7f532c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5332c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c19ec70 con 0x7f532c112b20 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5332c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c19edb0 con 0x7f532c076ac0 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5332c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c19ef70 con 0x7f532c0761a0 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f532be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c1a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f532be00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c1a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:59406/0 (socket says 172.21.15.6:59406) 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f532be00700 1 -- 172.21.15.6:0/1310468158 learned_addr learned my addr 172.21.15.6:0/1310468158 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f532b400700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f532c076ac0 0x7f532c1a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.739+0000 7f5330c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 0x7f532c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f532be00700 1 -- 172.21.15.6:0/1310468158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f532c076ac0 msgr2=0x7f532c1a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f532be00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f532c076ac0 0x7f532c1a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f532be00700 1 -- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 msgr2=0x7f532c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f532be00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 0x7f532c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f532be00700 1 -- 172.21.15.6:0/1310468158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0095f0 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f5330c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 0x7f532c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f532be00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c1a4450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f531c000c00 tx=0x7f531c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c01d070 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f531c00b610 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c19f1f0 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.740+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0173f0 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.741+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c1ab4f0 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.742+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c10dd40 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.745+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f531c0226e0 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.747+0000 7f5328c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f53100a2260 0x7f53100a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.747+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f531c0cf0a0 con 0x7f532c0761a0 2024-02-26T06:26:19.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.747+0000 7f532b400700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f53100a2260 0x7f53100a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.750+0000 7f532b400700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f53100a2260 0x7f53100a4720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f532c103430 tx=0x7f5318009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.751+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f531c05dd20 con 0x7f532c0761a0 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.988+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f532c19fb40 con 0x7f532c0761a0 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.988+0000 7f5328c00700 1 -- 172.21.15.6:0/1310468158 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f531c097f30 con 0x7f532c0761a0 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stdout:77309411406 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f53100a2260 msgr2=0x7f53100a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f53100a2260 0x7f53100a4720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f532c103430 tx=0x7f5318009260 comp rx=0 tx=0).stop 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 msgr2=0x7f532c1a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c1a4450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f531c000c00 tx=0x7f531c005740 comp rx=0 tx=0).stop 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 shutdown_connections 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f53100a2260 0x7f53100a4720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f532c112b20 0x7f532c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c0761a0 0x7f532c1a4450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 --2- 172.21.15.6:0/1310468158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f532c076ac0 0x7f532c1a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 >> 172.21.15.6:0/1310468158 conn(0x7f532c0ff970 msgr2=0x7f532c101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 shutdown_connections 2024-02-26T06:26:19.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:17.992+0000 7f5332c00700 1 -- 172.21.15.6:0/1310468158 wait complete. 2024-02-26T06:26:19.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:18 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1310468158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T06:26:19.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:18 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1310468158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T06:26:20.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:19 smithi006 ceph-mon[31790]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:20.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:19 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:19 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:19 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:19 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:19 smithi167 ceph-mon[36229]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:19 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:19 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:19 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:19 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:19 smithi090 ceph-mon[35681]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:19 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:19 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:19 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:19 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:22.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:21 smithi006 ceph-mon[31790]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:22.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:21 smithi167 ceph-mon[36229]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:22.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:21 smithi090 ceph-mon[35681]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:23.540 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:24.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:23 smithi006 ceph-mon[31790]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:24.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:23 smithi167 ceph-mon[36229]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:24.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:23 smithi090 ceph-mon[35681]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-02-26T06:26:24.430 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:26.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:25 smithi006 ceph-mon[31790]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 70 B/s, 0 keys/s, 1 objects/s recovering 2024-02-26T06:26:26.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411406 for osd.2 2024-02-26T06:26:26.233 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:26.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:25 smithi167 ceph-mon[36229]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 70 B/s, 0 keys/s, 1 objects/s recovering 2024-02-26T06:26:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:25 smithi090 ceph-mon[35681]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 70 B/s, 0 keys/s, 1 objects/s recovering 2024-02-26T06:26:27.992 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:27 smithi006 ceph-mon[31790]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 60 B/s, 0 keys/s, 1 objects/s recovering 2024-02-26T06:26:28.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:27 smithi167 ceph-mon[36229]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 60 B/s, 0 keys/s, 1 objects/s recovering 2024-02-26T06:26:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:27 smithi090 ceph-mon[35681]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 60 B/s, 0 keys/s, 1 objects/s recovering 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 -- 172.21.15.6:0/2615962249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70106b80 msgr2=0x7f1d70106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 --2- 172.21.15.6:0/2615962249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70106b80 0x7f1d70106f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d640098f0 tx=0x7f1d64009c00 comp rx=0 tx=0).stop 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 -- 172.21.15.6:0/2615962249 shutdown_connections 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 --2- 172.21.15.6:0/2615962249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70101530 0x7f1d7010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 --2- 172.21.15.6:0/2615962249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70106b80 0x7f1d70106f60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 --2- 172.21.15.6:0/2615962249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d70100b70 0x7f1d70100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 -- 172.21.15.6:0/2615962249 >> 172.21.15.6:0/2615962249 conn(0x7f1d70077aa0 msgr2=0x7f1d70077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.519+0000 7f1d77600700 1 -- 172.21.15.6:0/2615962249 shutdown_connections 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.520+0000 7f1d77600700 1 -- 172.21.15.6:0/2615962249 wait complete. 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 Processor -- start 2024-02-26T06:26:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 -- start start 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d70100b70 0x7f1d7019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 0x7f1d7019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70106b80 0x7f1d7019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7019f210 con 0x7f1d70106b80 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7019f380 con 0x7f1d70100b70 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.521+0000 7f1d77600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7019f540 con 0x7f1d70101530 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 0x7f1d7019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d74c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d70100b70 0x7f1d7019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 0x7f1d7019a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49154/0 (socket says 172.21.15.6:49154) 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 -- 172.21.15.6:0/1967102728 learned_addr learned my addr 172.21.15.6:0/1967102728 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d75600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70106b80 0x7f1d7019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 -- 172.21.15.6:0/1967102728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d70100b70 msgr2=0x7f1d7019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d70100b70 0x7f1d7019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 -- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70106b80 msgr2=0x7f1d7019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70106b80 0x7f1d7019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d6fe00700 1 -- 172.21.15.6:0/1967102728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64009520 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.522+0000 7f1d75600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70106b80 0x7f1d7019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.523+0000 7f1d6fe00700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 0x7f1d7019a870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9c0 tx=0x7f1d5c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.523+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00cbd0 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.523+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c00eeb0 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.523+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c005680 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.523+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7019f820 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.523+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d701a7080 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.524+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d70103e80 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.526+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f1d5c004750 con 0x7f1d70101530 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.528+0000 7f1d6d600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1d580a2260 0x7f1d580a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:28.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.528+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1d5c014070 con 0x7f1d70101530 2024-02-26T06:26:28.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.528+0000 7f1d74c00700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1d580a2260 0x7f1d580a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:28.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.529+0000 7f1d74c00700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1d580a2260 0x7f1d580a4720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009810 tx=0x7f1d64019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:28.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.532+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d5c096f40 con 0x7f1d70101530 2024-02-26T06:26:28.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.766+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1d70068930 con 0x7f1d70101530 2024-02-26T06:26:28.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.767+0000 7f1d6d600700 1 -- 172.21.15.6:0/1967102728 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d5c00cd30 con 0x7f1d70101530 2024-02-26T06:26:28.767 INFO:teuthology.orchestra.run.smithi006.stdout:219043332147 2024-02-26T06:26:28.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1d580a2260 msgr2=0x7f1d580a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:28.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1d580a2260 0x7f1d580a4720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009810 tx=0x7f1d64019040 comp rx=0 tx=0).stop 2024-02-26T06:26:28.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 msgr2=0x7f1d7019a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:28.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 0x7f1d7019a870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9c0 tx=0x7f1d5c00ecd0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 shutdown_connections 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1d580a2260 0x7f1d580a4720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d70106b80 0x7f1d7019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d70100b70 0x7f1d7019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 --2- 172.21.15.6:0/1967102728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d70101530 0x7f1d7019a870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 >> 172.21.15.6:0/1967102728 conn(0x7f1d70077aa0 msgr2=0x7f1d700ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 shutdown_connections 2024-02-26T06:26:28.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:28.770+0000 7f1d77600700 1 -- 172.21.15.6:0/1967102728 wait complete. 2024-02-26T06:26:29.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:28 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1967102728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T06:26:29.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:28 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1967102728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T06:26:29.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:28 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1967102728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T06:26:29.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 -- 172.21.15.6:0/4093044752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa41008a0 msgr2=0x7effa4100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 --2- 172.21.15.6:0/4093044752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa41008a0 0x7effa4100c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff94009990 tx=0x7eff94009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 -- 172.21.15.6:0/4093044752 shutdown_connections 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 --2- 172.21.15.6:0/4093044752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 0x7effa4110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 --2- 172.21.15.6:0/4093044752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa41008a0 0x7effa4100c80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 --2- 172.21.15.6:0/4093044752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa4101250 0x7effa410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 -- 172.21.15.6:0/4093044752 >> 172.21.15.6:0/4093044752 conn(0x7effa40fc540 msgr2=0x7effa40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 -- 172.21.15.6:0/4093044752 shutdown_connections 2024-02-26T06:26:29.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.978+0000 7effa8e00700 1 -- 172.21.15.6:0/4093044752 wait complete. 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.979+0000 7effa8e00700 1 Processor -- start 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.979+0000 7effa8e00700 1 -- start start 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 0x7effa419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 0x7effa419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa8e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 0x7effa419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa8e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa419f160 con 0x7effa4101250 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa8e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa419f2d0 con 0x7effa41008a0 2024-02-26T06:26:29.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa8e00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa419f490 con 0x7effa410e450 2024-02-26T06:26:29.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa2a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 0x7effa419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 0x7effa419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 0x7effa419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38016/0 (socket says 172.21.15.6:38016) 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa3400700 1 -- 172.21.15.6:0/2801204767 learned_addr learned my addr 172.21.15.6:0/2801204767 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa2a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 0x7effa419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34316/0 (socket says 172.21.15.6:34316) 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.980+0000 7effa3e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 0x7effa419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa2a00700 1 -- 172.21.15.6:0/2801204767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 msgr2=0x7effa419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa2a00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 0x7effa419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa2a00700 1 -- 172.21.15.6:0/2801204767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 msgr2=0x7effa419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa2a00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 0x7effa419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa2a00700 1 -- 172.21.15.6:0/2801204767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940095f0 con 0x7effa4101250 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa3400700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 0x7effa419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa3e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 0x7effa419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa2a00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 0x7effa419a810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff9000e9f0 tx=0x7eff9000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000c9c0 con 0x7effa4101250 2024-02-26T06:26:29.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff90010430 con 0x7effa4101250 2024-02-26T06:26:29.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa419f770 con 0x7effa4101250 2024-02-26T06:26:29.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.981+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90018680 con 0x7effa4101250 2024-02-26T06:26:29.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.982+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa41a6fd0 con 0x7effa4101250 2024-02-26T06:26:29.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.983+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa4109670 con 0x7effa4101250 2024-02-26T06:26:29.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.985+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7eff90010910 con 0x7effa4101250 2024-02-26T06:26:29.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.986+0000 7eff9be00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7eff880a21e0 0x7eff880a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:29.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.986+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7eff90014070 con 0x7effa4101250 2024-02-26T06:26:29.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.986+0000 7effa3400700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7eff880a21e0 0x7eff880a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:29.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.987+0000 7effa3400700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7eff880a21e0 0x7eff880a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff94006010 tx=0x7eff9400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:29.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:29.991+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff90052d20 con 0x7effa4101250 2024-02-26T06:26:30.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:29 smithi006 ceph-mon[31790]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:30.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.231+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7effa419b5e0 con 0x7effa4101250 2024-02-26T06:26:30.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.232+0000 7eff9be00700 1 -- 172.21.15.6:0/2801204767 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7eff90096e40 con 0x7effa4101250 2024-02-26T06:26:30.232 INFO:teuthology.orchestra.run.smithi006.stdout:309237645345 2024-02-26T06:26:30.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7eff880a21e0 msgr2=0x7eff880a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:30.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7eff880a21e0 0x7eff880a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff94006010 tx=0x7eff9400b610 comp rx=0 tx=0).stop 2024-02-26T06:26:30.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 msgr2=0x7effa419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:30.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 0x7effa419a810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff9000e9f0 tx=0x7eff9000ed00 comp rx=0 tx=0).stop 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 shutdown_connections 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7eff880a21e0 0x7eff880a46a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa410e450 0x7effa419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa41008a0 0x7effa419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 --2- 172.21.15.6:0/2801204767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa4101250 0x7effa419a810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.235+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 >> 172.21.15.6:0/2801204767 conn(0x7effa40fc540 msgr2=0x7effa410f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:30.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.236+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 shutdown_connections 2024-02-26T06:26:30.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.236+0000 7effa8e00700 1 -- 172.21.15.6:0/2801204767 wait complete. 2024-02-26T06:26:30.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:29 smithi167 ceph-mon[36229]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:30.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:29 smithi090 ceph-mon[35681]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:30 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2801204767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T06:26:31.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:30 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2801204767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T06:26:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:30 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2801204767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T06:26:31.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.970+0000 7f9616000700 1 -- 172.21.15.6:0/1367990759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610069920 msgr2=0x7f9610069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.970+0000 7f9616000700 1 --2- 172.21.15.6:0/1367990759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610069920 0x7f9610069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0099c0 tx=0x7f95fc009cd0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 -- 172.21.15.6:0/1367990759 shutdown_connections 2024-02-26T06:26:31.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 --2- 172.21.15.6:0/1367990759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 0x7f961010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 --2- 172.21.15.6:0/1367990759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610107cc0 0x7f96101080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 --2- 172.21.15.6:0/1367990759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610069920 0x7f9610069d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 -- 172.21.15.6:0/1367990759 >> 172.21.15.6:0/1367990759 conn(0x7f9610077aa0 msgr2=0x7f9610077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 -- 172.21.15.6:0/1367990759 shutdown_connections 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.971+0000 7f9616000700 1 -- 172.21.15.6:0/1367990759 wait complete. 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.972+0000 7f9616000700 1 Processor -- start 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 -- start start 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610069920 0x7f961019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 0x7f961019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610107cc0 0x7f961019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019f230 con 0x7f9610107cc0 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019f3a0 con 0x7f9610069920 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f9616000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019f560 con 0x7f961006a2c0 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f960fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610107cc0 0x7f961019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.973+0000 7f960ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 0x7f961019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960ea00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 0x7f961019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49198/0 (socket says 172.21.15.6:49198) 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960ea00700 1 -- 172.21.15.6:0/1364830102 learned_addr learned my addr 172.21.15.6:0/1364830102 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960f400700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610069920 0x7f961019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960fe00700 1 -- 172.21.15.6:0/1364830102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 msgr2=0x7f961019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960fe00700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 0x7f961019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960fe00700 1 -- 172.21.15.6:0/1364830102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610069920 msgr2=0x7f961019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960fe00700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610069920 0x7f961019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960fe00700 1 -- 172.21.15.6:0/1364830102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0095f0 con 0x7f9610107cc0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960f400700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610069920 0x7f961019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.974+0000 7f960fe00700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610107cc0 0x7f961019ec20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f960800efc0 tx=0x7f960800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.975+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608018070 con 0x7f9610107cc0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.975+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f960800f040 con 0x7f9610107cc0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.975+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961019f7e0 con 0x7f9610107cc0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.975+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800cd10 con 0x7f9610107cc0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.975+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96101a70a0 con 0x7f9610107cc0 2024-02-26T06:26:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.976+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9610068930 con 0x7f9610107cc0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.978+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f9608004ad0 con 0x7f9610107cc0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.980+0000 7f9607e00700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95f40a2260 0x7f95f40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.980+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f96080ce260 con 0x7f9610107cc0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.980+0000 7f960f400700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95f40a2260 0x7f95f40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.981+0000 7f960f400700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95f40a2260 0x7f95f40a4720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9600009fd0 tx=0x7f9600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:30.982+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96080970f0 con 0x7f9610107cc0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.213+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9610111ca0 con 0x7f9610107cc0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.214+0000 7f9607e00700 1 -- 172.21.15.6:0/1364830102 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f960809a7b0 con 0x7f9610107cc0 2024-02-26T06:26:31.568 INFO:teuthology.orchestra.run.smithi006.stdout:107374182473 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95f40a2260 msgr2=0x7f95f40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95f40a2260 0x7f95f40a4720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9600009fd0 tx=0x7f9600009260 comp rx=0 tx=0).stop 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610107cc0 msgr2=0x7f961019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610107cc0 0x7f961019ec20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f960800efc0 tx=0x7f960800c3a0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 shutdown_connections 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95f40a2260 0x7f95f40a4720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9610107cc0 0x7f961019ec20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9610069920 0x7f961019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.216+0000 7f9616000700 1 --2- 172.21.15.6:0/1364830102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961006a2c0 0x7f961019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.217+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 >> 172.21.15.6:0/1364830102 conn(0x7f9610077aa0 msgr2=0x7f961010ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:31.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.217+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 shutdown_connections 2024-02-26T06:26:31.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.217+0000 7f9616000700 1 -- 172.21.15.6:0/1364830102 wait complete. 2024-02-26T06:26:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 -- 172.21.15.6:0/3113064000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc474073ea0 msgr2=0x7fc474074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 --2- 172.21.15.6:0/3113064000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc474073ea0 0x7fc474074280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc4640099c0 tx=0x7fc464009cd0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 -- 172.21.15.6:0/3113064000 shutdown_connections 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 --2- 172.21.15.6:0/3113064000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc474112e40 0x7fc474115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 --2- 172.21.15.6:0/3113064000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc474073ea0 0x7fc474074280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 --2- 172.21.15.6:0/3113064000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4740747c0 0x7fc474112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 -- 172.21.15.6:0/3113064000 >> 172.21.15.6:0/3113064000 conn(0x7fc474078da0 msgr2=0x7fc4740791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 -- 172.21.15.6:0/3113064000 shutdown_connections 2024-02-26T06:26:31.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.973+0000 7fc47b000700 1 -- 172.21.15.6:0/3113064000 wait complete. 2024-02-26T06:26:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.975+0000 7fc47b000700 1 Processor -- start 2024-02-26T06:26:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.975+0000 7fc47b000700 1 -- start start 2024-02-26T06:26:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.975+0000 7fc47b000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 0x7fc4741a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc47b000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4740747c0 0x7fc4741a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc47b000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc474112e40 0x7fc47419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc47b000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47419ec80 con 0x7fc4740747c0 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc47b000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47419edc0 con 0x7fc474112e40 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc47b000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47419ef80 con 0x7fc474073ea0 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 0x7fc4741a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4740747c0 0x7fc4741a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 0x7fc4741a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49214/0 (socket says 172.21.15.6:49214) 2024-02-26T06:26:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 -- 172.21.15.6:0/333470737 learned_addr learned my addr 172.21.15.6:0/333470737 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc479000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc474112e40 0x7fc47419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 -- 172.21.15.6:0/333470737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc474112e40 msgr2=0x7fc47419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc474112e40 0x7fc47419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 -- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4740747c0 msgr2=0x7fc4741a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4740747c0 0x7fc4741a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473e00700 1 -- 172.21.15.6:0/333470737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640095f0 con 0x7fc474073ea0 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.976+0000 7fc473400700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4740747c0 0x7fc4741a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.977+0000 7fc473e00700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 0x7fc4741a4460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4640098e0 tx=0x7fc464005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.977+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46401d070 con 0x7fc474073ea0 2024-02-26T06:26:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.977+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47419f200 con 0x7fc474073ea0 2024-02-26T06:26:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.977+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc464003680 con 0x7fc474073ea0 2024-02-26T06:26:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.977+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464022c90 con 0x7fc474073ea0 2024-02-26T06:26:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.977+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4741ab500 con 0x7fc474073ea0 2024-02-26T06:26:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.978+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc474075590 con 0x7fc474073ea0 2024-02-26T06:26:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.980+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fc464051840 con 0x7fc474073ea0 2024-02-26T06:26:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.981+0000 7fc470c00700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4580a21e0 0x7fc4580a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.982+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc4640cf2d0 con 0x7fc474073ea0 2024-02-26T06:26:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.982+0000 7fc473400700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4580a21e0 0x7fc4580a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.982+0000 7fc473400700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4580a21e0 0x7fc4580a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc47410f7c0 tx=0x7fc4600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:31.984+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc464098160 con 0x7fc474073ea0 2024-02-26T06:26:32.076 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:31 smithi006 ceph-mon[31790]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:32.076 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:31 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1364830102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T06:26:32.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.204+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fc47419fb80 con 0x7fc474073ea0 2024-02-26T06:26:32.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.205+0000 7fc470c00700 1 -- 172.21.15.6:0/333470737 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc46409b820 con 0x7fc474073ea0 2024-02-26T06:26:32.206 INFO:teuthology.orchestra.run.smithi006.stdout:287762808871 2024-02-26T06:26:32.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4580a21e0 msgr2=0x7fc4580a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4580a21e0 0x7fc4580a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc47410f7c0 tx=0x7fc4600073d0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 msgr2=0x7fc4741a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 0x7fc4741a4460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4640098e0 tx=0x7fc464005740 comp rx=0 tx=0).stop 2024-02-26T06:26:32.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 shutdown_connections 2024-02-26T06:26:32.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4580a21e0 0x7fc4580a46a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc474112e40 0x7fc47419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc474073ea0 0x7fc4741a4460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 --2- 172.21.15.6:0/333470737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4740747c0 0x7fc4741a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 >> 172.21.15.6:0/333470737 conn(0x7fc474078da0 msgr2=0x7fc474111c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 shutdown_connections 2024-02-26T06:26:32.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.208+0000 7fc47b000700 1 -- 172.21.15.6:0/333470737 wait complete. 2024-02-26T06:26:32.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:31 smithi167 ceph-mon[36229]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:32.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:31 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1364830102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T06:26:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:31 smithi090 ceph-mon[35681]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:31 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1364830102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 -- 172.21.15.6:0/84102017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0101cc0 msgr2=0x7fdbb0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/84102017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0101cc0 0x7fdbb0102140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009920 tx=0x7fdb9c009c30 comp rx=0 tx=0).stop 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 -- 172.21.15.6:0/84102017 shutdown_connections 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/84102017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0102680 0x7fdbb010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/84102017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb0107ca0 0x7fdbb0108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/84102017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0101cc0 0x7fdbb0102140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 -- 172.21.15.6:0/84102017 >> 172.21.15.6:0/84102017 conn(0x7fdbb0078da0 msgr2=0x7fdbb00791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 -- 172.21.15.6:0/84102017 shutdown_connections 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.662+0000 7fdbb4a00700 1 -- 172.21.15.6:0/84102017 wait complete. 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.663+0000 7fdbb4a00700 1 Processor -- start 2024-02-26T06:26:32.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.663+0000 7fdbb4a00700 1 -- start start 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbb4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 0x7fdbb019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbb4a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb0102680 0x7fdbb019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbb4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0107ca0 0x7fdbb019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbb4a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019f390 con 0x7fdbb0101cc0 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbb4a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019f500 con 0x7fdbb0107ca0 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbb4a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019f6c0 con 0x7fdbb0102680 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 0x7fdbb019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 0x7fdbb019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34376/0 (socket says 172.21.15.6:34376) 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 -- 172.21.15.6:0/2560545306 learned_addr learned my addr 172.21.15.6:0/2560545306 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbad400700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb0102680 0x7fdbb019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbae800700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0107ca0 0x7fdbb019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 -- 172.21.15.6:0/2560545306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb0102680 msgr2=0x7fdbb019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb0102680 0x7fdbb019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 -- 172.21.15.6:0/2560545306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0107ca0 msgr2=0x7fdbb019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0107ca0 0x7fdbb019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.664+0000 7fdbade00700 1 -- 172.21.15.6:0/2560545306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c009520 con 0x7fdbb0101cc0 2024-02-26T06:26:32.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.665+0000 7fdbade00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 0x7fdbb019a4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e9c0 tx=0x7fdba000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.665+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0004a30 con 0x7fdbb0101cc0 2024-02-26T06:26:32.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.665+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb019f9a0 con 0x7fdbb0101cc0 2024-02-26T06:26:32.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.665+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdba000cca0 con 0x7fdbb0101cc0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.665+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0005580 con 0x7fdbb0101cc0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.665+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb01a7200 con 0x7fdbb0101cc0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.666+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb0068930 con 0x7fdbb0101cc0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.668+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fdba0004c20 con 0x7fdbb0101cc0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.670+0000 7fdba6a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdb940a2260 0x7fdb940a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.670+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fdba0014070 con 0x7fdbb0101cc0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.670+0000 7fdbad400700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdb940a2260 0x7fdb940a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.671+0000 7fdbad400700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdb940a2260 0x7fdb940a4720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0098f0 tx=0x7fdb9c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:32.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.674+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdba00d4080 con 0x7fdbb0101cc0 2024-02-26T06:26:32.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.907+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdbb019b810 con 0x7fdbb0101cc0 2024-02-26T06:26:32.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.908+0000 7fdba6a00700 1 -- 172.21.15.6:0/2560545306 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdba0096c80 con 0x7fdbb0101cc0 2024-02-26T06:26:32.908 INFO:teuthology.orchestra.run.smithi006.stdout:55834574933 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdb940a2260 msgr2=0x7fdb940a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdb940a2260 0x7fdb940a4720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0098f0 tx=0x7fdb9c019040 comp rx=0 tx=0).stop 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 msgr2=0x7fdbb019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 0x7fdbb019a4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e9c0 tx=0x7fdba000ecd0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 shutdown_connections 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdb940a2260 0x7fdb940a4720 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbb0107ca0 0x7fdbb019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbb0101cc0 0x7fdbb019a4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 --2- 172.21.15.6:0/2560545306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb0102680 0x7fdbb019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.910+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 >> 172.21.15.6:0/2560545306 conn(0x7fdbb0078da0 msgr2=0x7fdbb00741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.911+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 shutdown_connections 2024-02-26T06:26:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.911+0000 7fdbb4a00700 1 -- 172.21.15.6:0/2560545306 wait complete. 2024-02-26T06:26:33.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:32 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/333470737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T06:26:33.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:32 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2560545306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T06:26:33.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:32 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/333470737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T06:26:33.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:32 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2560545306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T06:26:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:32 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/333470737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T06:26:33.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:32 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2560545306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T06:26:34.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:33 smithi167 ceph-mon[36229]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:26:34.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:33 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3186841707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 -- 172.21.15.6:0/311514263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 msgr2=0x7f84a0100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 --2- 172.21.15.6:0/311514263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a0100c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84900098f0 tx=0x7f8490009c00 comp rx=0 tx=0).stop 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 -- 172.21.15.6:0/311514263 shutdown_connections 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 --2- 172.21.15.6:0/311514263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a010e450 0x7f84a0110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 --2- 172.21.15.6:0/311514263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a0100c80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 --2- 172.21.15.6:0/311514263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a0101250 0x7f84a010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.797+0000 7f84a5400700 1 -- 172.21.15.6:0/311514263 >> 172.21.15.6:0/311514263 conn(0x7f84a00fc540 msgr2=0x7f84a00fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.798+0000 7f84a5400700 1 -- 172.21.15.6:0/311514263 shutdown_connections 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.798+0000 7f84a5400700 1 -- 172.21.15.6:0/311514263 wait complete. 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 Processor -- start 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 -- start start 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0101250 0x7f84a019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a010e450 0x7f84a019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a019f1d0 con 0x7f84a010e450 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a019f340 con 0x7f84a01008a0 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a5400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a019f500 con 0x7f84a0101250 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f849fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.799+0000 7f84a4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a010e450 0x7f84a019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38088/0 (socket says 172.21.15.6:38088) 2024-02-26T06:26:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 -- 172.21.15.6:0/3186841707 learned_addr learned my addr 172.21.15.6:0/3186841707 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849f400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0101250 0x7f84a019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 -- 172.21.15.6:0/3186841707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0101250 msgr2=0x7f84a019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0101250 0x7f84a019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 -- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a010e450 msgr2=0x7f84a019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a010e450 0x7f84a019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 -- 172.21.15.6:0/3186841707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c0095f0 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f84a4a00700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a010e450 0x7f84a019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849fe00700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a019a2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8490009230 tx=0x7f8490005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.800+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849001c070 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.801+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f849000ba40 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.801+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8490009520 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.801+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490003d20 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.801+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a01a6e70 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.802+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a0109670 con 0x7f84a01008a0 2024-02-26T06:26:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.804+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f849000b560 con 0x7f84a01008a0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.805+0000 7f849cc00700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f84840a2260 0x7f84840a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.805+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f84900cec80 con 0x7f84a01008a0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.805+0000 7f849f400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f84840a2260 0x7f84840a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.806+0000 7f849f400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f84840a2260 0x7f84840a4720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84a0110d30 tx=0x7f848c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:32.807+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8490097b70 con 0x7f84a01008a0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.051+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f84a006a200 con 0x7f84a01008a0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.052+0000 7f849cc00700 1 -- 172.21.15.6:0/3186841707 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f849009b230 con 0x7f84a01008a0 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f84840a2260 msgr2=0x7f84840a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f84840a2260 0x7f84840a4720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84a0110d30 tx=0x7f848c009310 comp rx=0 tx=0).stop 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 msgr2=0x7f84a019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a019a2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8490009230 tx=0x7f8490005040 comp rx=0 tx=0).stop 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 shutdown_connections 2024-02-26T06:26:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f84840a2260 0x7f84840a4720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84a010e450 0x7f84a019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a01008a0 0x7f84a019a2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 --2- 172.21.15.6:0/3186841707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0101250 0x7f84a019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 >> 172.21.15.6:0/3186841707 conn(0x7f84a00fc540 msgr2=0x7f84a00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 shutdown_connections 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:33.055+0000 7f84a5400700 1 -- 172.21.15.6:0/3186841707 wait complete. 2024-02-26T06:26:34.259 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:33 smithi006 ceph-mon[31790]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:26:34.259 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:33 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3186841707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T06:26:34.259 INFO:teuthology.orchestra.run.smithi006.stdout:141733920837 2024-02-26T06:26:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:33 smithi090 ceph-mon[35681]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:26:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:33 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3186841707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T06:26:34.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920832 got 141733920837 for osd.4 2024-02-26T06:26:34.985 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.459+0000 7f2fb9600700 1 -- 172.21.15.6:0/3043513116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406af20 msgr2=0x7f2fb4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.459+0000 7f2fb9600700 1 --2- 172.21.15.6:0/3043513116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406af20 0x7f2fb4106920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009920 tx=0x7f2fa0009c30 comp rx=0 tx=0).stop 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.459+0000 7f2fb9600700 1 -- 172.21.15.6:0/3043513116 shutdown_connections 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.459+0000 7f2fb9600700 1 --2- 172.21.15.6:0/3043513116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb4107050 0x7f2fb410ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.459+0000 7f2fb9600700 1 --2- 172.21.15.6:0/3043513116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb406a570 0x7f2fb406a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.459+0000 7f2fb9600700 1 --2- 172.21.15.6:0/3043513116 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406af20 0x7f2fb4106920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.460+0000 7f2fb9600700 1 -- 172.21.15.6:0/3043513116 >> 172.21.15.6:0/3043513116 conn(0x7f2fb4077a30 msgr2=0x7f2fb4077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.460+0000 7f2fb9600700 1 -- 172.21.15.6:0/3043513116 shutdown_connections 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.460+0000 7f2fb9600700 1 -- 172.21.15.6:0/3043513116 wait complete. 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.461+0000 7f2fb9600700 1 Processor -- start 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.461+0000 7f2fb9600700 1 -- start start 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.461+0000 7f2fb9600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406a570 0x7f2fb419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.461+0000 7f2fb9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb406af20 0x7f2fb419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.461+0000 7f2fb9600700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb4107050 0x7f2fb419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb9600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419f330 con 0x7f2fb406a570 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb9600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419f4a0 con 0x7f2fb406af20 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb9600700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419f660 con 0x7f2fb4107050 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406a570 0x7f2fb419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb406af20 0x7f2fb419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406a570 0x7f2fb419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34390/0 (socket says 172.21.15.6:34390) 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2a00700 1 -- 172.21.15.6:0/2761470075 learned_addr learned my addr 172.21.15.6:0/2761470075 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:35.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb3400700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb4107050 0x7f2fb419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2000700 1 -- 172.21.15.6:0/2761470075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb4107050 msgr2=0x7f2fb419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2000700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb4107050 0x7f2fb419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2000700 1 -- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406a570 msgr2=0x7f2fb419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2000700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406a570 0x7f2fb419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.462+0000 7f2fb2000700 1 -- 172.21.15.6:0/2761470075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0009520 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.463+0000 7f2fb2000700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb406af20 0x7f2fb419a990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0000c00 tx=0x7f2fa000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.463+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa001c070 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.463+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa0005520 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.463+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb419f8e0 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.463+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0004930 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.463+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb41a71a0 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.464+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb41033d0 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.466+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f2fa0020070 con 0x7f2fb406af20 2024-02-26T06:26:35.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.467+0000 7f2fab400700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f980a21e0 0x7f2f980a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:35.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.468+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2fa00cecd0 con 0x7f2fb406af20 2024-02-26T06:26:35.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.468+0000 7f2fb2a00700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f980a21e0 0x7f2f980a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:35.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.468+0000 7f2fb2a00700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f980a21e0 0x7f2f980a46a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005950 tx=0x7f2fa4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:35.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.470+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2fa0097b60 con 0x7f2fb406af20 2024-02-26T06:26:35.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.702+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f2fb419b810 con 0x7f2fb406af20 2024-02-26T06:26:35.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.703+0000 7f2fab400700 1 -- 172.21.15.6:0/2761470075 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2fa009b220 con 0x7f2fb406af20 2024-02-26T06:26:35.703 INFO:teuthology.orchestra.run.smithi006.stdout:343597383711 2024-02-26T06:26:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f980a21e0 msgr2=0x7f2f980a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f980a21e0 0x7f2f980a46a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005950 tx=0x7f2fa4009310 comp rx=0 tx=0).stop 2024-02-26T06:26:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb406af20 msgr2=0x7f2fb419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb406af20 0x7f2fb419a990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0000c00 tx=0x7f2fa000fa00 comp rx=0 tx=0).stop 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 shutdown_connections 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2f980a21e0 0x7f2f980a46a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb4107050 0x7f2fb419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fb406a570 0x7f2fb419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 --2- 172.21.15.6:0/2761470075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb406af20 0x7f2fb419a990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 >> 172.21.15.6:0/2761470075 conn(0x7f2fb4077a30 msgr2=0x7f2fb410cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 shutdown_connections 2024-02-26T06:26:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:35.706+0000 7f2fb9600700 1 -- 172.21.15.6:0/2761470075 wait complete. 2024-02-26T06:26:36.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:35 smithi006 ceph-mon[31790]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:36.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:35 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2761470075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T06:26:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:35 smithi167 ceph-mon[36229]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:35 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2761470075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T06:26:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:35 smithi090 ceph-mon[35681]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:35 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2761470075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T06:26:37.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808871 for osd.9 2024-02-26T06:26:37.526 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:37.657 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332144 got 219043332147 for osd.7 2024-02-26T06:26:37.658 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:38.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:37 smithi006 ceph-mon[31790]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:26:38.204 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383711 for osd.11 2024-02-26T06:26:38.205 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:37 smithi167 ceph-mon[36229]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:26:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:37 smithi090 ceph-mon[35681]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:26:38.576 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645345 for osd.10 2024-02-26T06:26:38.576 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:39.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 -- 172.21.15.6:0/2737619717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4101f20 msgr2=0x7fb9a4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/2737619717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4101f20 0x7fb9a4102300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb994009990 tx=0x7fb994009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 -- 172.21.15.6:0/2737619717 shutdown_connections 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/2737619717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4106ff0 0x7fb9a410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/2737619717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4101f20 0x7fb9a4102300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/2737619717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a41028d0 0x7fb9a4106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 -- 172.21.15.6:0/2737619717 >> 172.21.15.6:0/2737619717 conn(0x7fb9a4078dd0 msgr2=0x7fb9a40791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.918+0000 7fb9a8c00700 1 -- 172.21.15.6:0/2737619717 shutdown_connections 2024-02-26T06:26:39.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.920+0000 7fb9a8c00700 1 -- 172.21.15.6:0/2737619717 wait complete. 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 Processor -- start 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 -- start start 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4101f20 0x7fb9a41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a41028d0 0x7fb9a41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4106ff0 0x7fb9a419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419d130 con 0x7fb9a4101f20 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419d2a0 con 0x7fb9a4106ff0 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a8c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419d460 con 0x7fb9a41028d0 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4101f20 0x7fb9a41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4106ff0 0x7fb9a419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:39.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4101f20 0x7fb9a41982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38076/0 (socket says 172.21.15.6:38076) 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a1e00700 1 -- 172.21.15.6:0/4294112895 learned_addr learned my addr 172.21.15.6:0/4294112895 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a1400700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a41028d0 0x7fb9a41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a2800700 1 -- 172.21.15.6:0/4294112895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a41028d0 msgr2=0x7fb9a41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a2800700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a41028d0 0x7fb9a41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a2800700 1 -- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4101f20 msgr2=0x7fb9a41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a2800700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4101f20 0x7fb9a41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.921+0000 7fb9a2800700 1 -- 172.21.15.6:0/4294112895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9940095f0 con 0x7fb9a4106ff0 2024-02-26T06:26:39.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.922+0000 7fb9a2800700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4106ff0 0x7fb9a419cb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00efc0 tx=0x7fb99c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.922+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c018070 con 0x7fb9a4106ff0 2024-02-26T06:26:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.922+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb99c00f040 con 0x7fb9a4106ff0 2024-02-26T06:26:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.922+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00cd20 con 0x7fb9a4106ff0 2024-02-26T06:26:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.922+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a419d6e0 con 0x7fb9a4106ff0 2024-02-26T06:26:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.922+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a41a4fa0 con 0x7fb9a4106ff0 2024-02-26T06:26:39.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.924+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a410a220 con 0x7fb9a4106ff0 2024-02-26T06:26:39.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.925+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb99c003a00 con 0x7fb9a4106ff0 2024-02-26T06:26:39.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.928+0000 7fb99aa00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9880a2260 0x7fb9880a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:39.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.928+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb99c0cde60 con 0x7fb9a4106ff0 2024-02-26T06:26:39.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.928+0000 7fb9a1e00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9880a2260 0x7fb9880a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:39.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.929+0000 7fb9a1e00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9880a2260 0x7fb9880a4720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb994005cd0 tx=0x7fb994005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:39.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:39.929+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb99c096cf0 con 0x7fb9a4106ff0 2024-02-26T06:26:40.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.164+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9a4199610 con 0x7fb9a4106ff0 2024-02-26T06:26:40.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.165+0000 7fb99aa00700 1 -- 172.21.15.6:0/4294112895 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb99c003cb0 con 0x7fb9a4106ff0 2024-02-26T06:26:40.165 INFO:teuthology.orchestra.run.smithi006.stdout:34359738460 2024-02-26T06:26:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9880a2260 msgr2=0x7fb9880a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9880a2260 0x7fb9880a4720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb994005cd0 tx=0x7fb994005c60 comp rx=0 tx=0).stop 2024-02-26T06:26:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4106ff0 msgr2=0x7fb9a419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4106ff0 0x7fb9a419cb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00efc0 tx=0x7fb99c00c3a0 comp rx=0 tx=0).stop 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 shutdown_connections 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9880a2260 0x7fb9880a4720 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4106ff0 0x7fb9a419cb20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a4101f20 0x7fb9a41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/4294112895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a41028d0 0x7fb9a41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 >> 172.21.15.6:0/4294112895 conn(0x7fb9a4078dd0 msgr2=0x7fb9a4106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 shutdown_connections 2024-02-26T06:26:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:40.168+0000 7fb9a8c00700 1 -- 172.21.15.6:0/4294112895 wait complete. 2024-02-26T06:26:40.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:39 smithi006 ceph-mon[31790]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:40.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:39 smithi167 ceph-mon[36229]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:39 smithi090 ceph-mon[35681]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:40.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182473 for osd.3 2024-02-26T06:26:40.628 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:41.142 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:40 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4294112895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T06:26:41.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:40 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4294112895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T06:26:41.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:40 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4294112895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T06:26:41.848 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738460 for osd.0 2024-02-26T06:26:41.848 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 -- 172.21.15.6:0/2561117463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee88108e90 msgr2=0x7fee8810cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 --2- 172.21.15.6:0/2561117463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee88108e90 0x7fee8810cee0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee74009990 tx=0x7fee74009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 -- 172.21.15.6:0/2561117463 shutdown_connections 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 --2- 172.21.15.6:0/2561117463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee8810d5c0 0x7fee88115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 --2- 172.21.15.6:0/2561117463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee881084e0 0x7fee881088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 --2- 172.21.15.6:0/2561117463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee88108e90 0x7fee8810cee0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 -- 172.21.15.6:0/2561117463 >> 172.21.15.6:0/2561117463 conn(0x7fee88075440 msgr2=0x7fee88077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 -- 172.21.15.6:0/2561117463 shutdown_connections 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.556+0000 7fee8fc00700 1 -- 172.21.15.6:0/2561117463 wait complete. 2024-02-26T06:26:41.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.557+0000 7fee8fc00700 1 Processor -- start 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.557+0000 7fee8fc00700 1 -- start start 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.557+0000 7fee8fc00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 0x7fee881a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee88108e90 0x7fee881a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8fc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee8810d5c0 0x7fee8819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8fc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8819ec40 con 0x7fee8810d5c0 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8819ed80 con 0x7fee88108e90 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8fc00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8819ef40 con 0x7fee881084e0 2024-02-26T06:26:41.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8d200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 0x7fee881a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8dc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee8810d5c0 0x7fee8819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8d200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 0x7fee881a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36350/0 (socket says 172.21.15.6:36350) 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8d200700 1 -- 172.21.15.6:0/592295656 learned_addr learned my addr 172.21.15.6:0/592295656 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee87e00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee88108e90 0x7fee881a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8dc00700 1 -- 172.21.15.6:0/592295656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 msgr2=0x7fee881a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8dc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 0x7fee881a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8dc00700 1 -- 172.21.15.6:0/592295656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee88108e90 msgr2=0x7fee881a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8dc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee88108e90 0x7fee881a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8dc00700 1 -- 172.21.15.6:0/592295656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee740095f0 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.558+0000 7fee8d200700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 0x7fee881a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.559+0000 7fee8dc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee8810d5c0 0x7fee8819e5e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee8000efc0 tx=0x7fee8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.559+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80018070 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.559+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8819f1c0 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.559+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee8000f040 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.559+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8000cd20 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.559+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee881ab4c0 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.560+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee881108e0 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.562+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fee80004ad0 con 0x7fee8810d5c0 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.563+0000 7fee85600700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee700a21e0 0x7fee700a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:41.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.564+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fee800cd920 con 0x7fee8810d5c0 2024-02-26T06:26:41.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.564+0000 7fee8d200700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee700a21e0 0x7fee700a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:41.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.565+0000 7fee8d200700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee700a21e0 0x7fee700a46a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee7c011550 tx=0x7fee7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:41.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.568+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee800967b0 con 0x7fee8810d5c0 2024-02-26T06:26:41.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.794+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fee8819fb10 con 0x7fee8810d5c0 2024-02-26T06:26:41.943 INFO:teuthology.orchestra.run.smithi006.stdout:253403070512 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.795+0000 7fee85600700 1 -- 172.21.15.6:0/592295656 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee80099e70 con 0x7fee8810d5c0 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee700a21e0 msgr2=0x7fee700a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee700a21e0 0x7fee700a46a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee7c011550 tx=0x7fee7c009260 comp rx=0 tx=0).stop 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee8810d5c0 msgr2=0x7fee8819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee8810d5c0 0x7fee8819e5e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee8000efc0 tx=0x7fee8000c3a0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 shutdown_connections 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fee700a21e0 0x7fee700a46a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee8810d5c0 0x7fee8819e5e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee881084e0 0x7fee881a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 --2- 172.21.15.6:0/592295656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee88108e90 0x7fee881a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:41.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.798+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 >> 172.21.15.6:0/592295656 conn(0x7fee88075440 msgr2=0x7fee88075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:41.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.799+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 shutdown_connections 2024-02-26T06:26:41.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:41.799+0000 7fee8fc00700 1 -- 172.21.15.6:0/592295656 wait complete. 2024-02-26T06:26:42.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:41 smithi006 ceph-mon[31790]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:42.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:41 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/592295656' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T06:26:42.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:41 smithi167 ceph-mon[36229]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:42.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:41 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/592295656' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T06:26:42.311 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574933 for osd.1 2024-02-26T06:26:42.311 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:41 smithi090 ceph-mon[35681]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:26:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:41 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/592295656' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 -- 172.21.15.6:0/1830040613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc107cc0 msgr2=0x7f0edc1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/1830040613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc107cc0 0x7f0edc1080a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0099c0 tx=0x7f0ecc009cd0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 -- 172.21.15.6:0/1830040613 shutdown_connections 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/1830040613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0edc06a2c0 0x7f0edc10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/1830040613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc107cc0 0x7f0edc1080a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/1830040613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc069920 0x7f0edc069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 -- 172.21.15.6:0/1830040613 >> 172.21.15.6:0/1830040613 conn(0x7f0edc077aa0 msgr2=0x7f0edc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 -- 172.21.15.6:0/1830040613 shutdown_connections 2024-02-26T06:26:43.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.108+0000 7f0ee1c00700 1 -- 172.21.15.6:0/1830040613 wait complete. 2024-02-26T06:26:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.109+0000 7f0ee1c00700 1 Processor -- start 2024-02-26T06:26:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.109+0000 7f0ee1c00700 1 -- start start 2024-02-26T06:26:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0ee1c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 0x7f0edc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0ee1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc06a2c0 0x7f0edc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0ee1c00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0edc107cc0 0x7f0edc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0ee1c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19f290 con 0x7f0edc069920 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0ee1c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19f400 con 0x7f0edc06a2c0 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0ee1c00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19f5c0 con 0x7f0edc107cc0 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0edaa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 0x7f0edc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0edaa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 0x7f0edc19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38138/0 (socket says 172.21.15.6:38138) 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0edaa00700 1 -- 172.21.15.6:0/604175994 learned_addr learned my addr 172.21.15.6:0/604175994 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0eda000700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc06a2c0 0x7f0edc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.110+0000 7f0edb400700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0edc107cc0 0x7f0edc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0eda000700 1 -- 172.21.15.6:0/604175994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0edc107cc0 msgr2=0x7f0edc19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0eda000700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0edc107cc0 0x7f0edc19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0eda000700 1 -- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 msgr2=0x7f0edc19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0eda000700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 0x7f0edc19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0eda000700 1 -- 172.21.15.6:0/604175994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0095f0 con 0x7f0edc06a2c0 2024-02-26T06:26:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0edaa00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 0x7f0edc19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0eda000700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc06a2c0 0x7f0edc19a8f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800e9f0 tx=0x7f0ec800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800c9c0 con 0x7f0edc06a2c0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec8010430 con 0x7f0edc06a2c0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8018680 con 0x7f0edc06a2c0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.111+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc19f8a0 con 0x7f0edc06a2c0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.112+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc1a7100 con 0x7f0edc06a2c0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.113+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc068930 con 0x7f0edc06a2c0 2024-02-26T06:26:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.114+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0ec8010910 con 0x7f0edc06a2c0 2024-02-26T06:26:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.116+0000 7f0ed3400700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0ec00a21d0 0x7f0ec00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.116+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0ec8014070 con 0x7f0edc06a2c0 2024-02-26T06:26:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.116+0000 7f0edaa00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0ec00a21d0 0x7f0ec00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.117+0000 7f0edaa00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0ec00a21d0 0x7f0ec00a4690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc000c00 tx=0x7f0ecc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:43.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.118+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ec8096d10 con 0x7f0edc06a2c0 2024-02-26T06:26:43.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 -- 172.21.15.6:0/1440249059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 msgr2=0x7f28a4096650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 --2- 172.21.15.6:0/1440249059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a4096650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f28a00099c0 tx=0x7f28a0009cd0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 -- 172.21.15.6:0/1440249059 shutdown_connections 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 --2- 172.21.15.6:0/1440249059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 0x7f28a40a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 --2- 172.21.15.6:0/1440249059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a4096650 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 --2- 172.21.15.6:0/1440249059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 0x7f28a409a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.008+0000 7f28b1400700 1 -- 172.21.15.6:0/1440249059 >> 172.21.15.6:0/1440249059 conn(0x7f28a400b8e0 msgr2=0x7f28a400bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.009+0000 7f28b1400700 1 -- 172.21.15.6:0/1440249059 shutdown_connections 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.009+0000 7f28b1400700 1 -- 172.21.15.6:0/1440249059 wait complete. 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 Processor -- start 2024-02-26T06:26:43.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 -- start start 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a412e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 0x7f28a412ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 0x7f28a4132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41333d0 con 0x7f28a4096270 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4133540 con 0x7f28a409b100 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.010+0000 7f28b1400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4133700 con 0x7f28a4096c20 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28ab400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 0x7f28a412ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a412e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 0x7f28a4132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28ab400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 0x7f28a412ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36366/0 (socket says 172.21.15.6:36366) 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28ab400700 1 -- 172.21.15.6:0/2147424498 learned_addr learned my addr 172.21.15.6:0/2147424498 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a412e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38120/0 (socket says 172.21.15.6:38120) 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 -- 172.21.15.6:0/2147424498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 msgr2=0x7f28a412ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 0x7f28a412ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 -- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 msgr2=0x7f28a412e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a412e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 -- 172.21.15.6:0/2147424498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898009520 con 0x7f28a409b100 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.011+0000 7f28b0a00700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 0x7f28a4132dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0686d0 tx=0x7f28ac06b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.012+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac073ea0 con 0x7f28a409b100 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.012+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28ac07dc30 con 0x7f28a409b100 2024-02-26T06:26:43.299 INFO:teuthology.orchestra.run.smithi006.stdout:197568495675 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.012+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac07ed10 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.012+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a00095f0 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.012+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a413b070 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.013+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a4005aa0 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.015+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f28ac07e460 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.016+0000 7f28a8c00700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28940a21d0 0x7f28940a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.016+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f28ac139860 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.016+0000 7f28abe00700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28940a21d0 0x7f28940a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.017+0000 7f28abe00700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28940a21d0 0x7f28940a4690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28a000b580 tx=0x7f28a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.020+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28ac102850 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.258+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f28a40a5e80 con 0x7f28a409b100 2024-02-26T06:26:43.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.259+0000 7f28a8c00700 1 -- 172.21.15.6:0/2147424498 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28ac105f10 con 0x7f28a409b100 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.261+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28940a21d0 msgr2=0x7f28940a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.261+0000 7f28b1400700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28940a21d0 0x7f28940a4690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28a000b580 tx=0x7f28a0005fb0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.261+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 msgr2=0x7f28a4132dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 0x7f28a4132dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0686d0 tx=0x7f28ac06b330 comp rx=0 tx=0).stop 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 shutdown_connections 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f28940a21d0 0x7f28940a4690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28a409b100 0x7f28a4132dc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28a4096270 0x7f28a412e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 --2- 172.21.15.6:0/2147424498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4096c20 0x7f28a412ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 >> 172.21.15.6:0/2147424498 conn(0x7f28a400b8e0 msgr2=0x7f28a4095800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 shutdown_connections 2024-02-26T06:26:43.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.262+0000 7f28b1400700 1 -- 172.21.15.6:0/2147424498 wait complete. 2024-02-26T06:26:43.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.349+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0edc111ce0 con 0x7f0edc06a2c0 2024-02-26T06:26:43.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.350+0000 7f0ed3400700 1 -- 172.21.15.6:0/604175994 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ec80105a0 con 0x7f0edc06a2c0 2024-02-26T06:26:43.349 INFO:teuthology.orchestra.run.smithi006.stdout:163208757314 2024-02-26T06:26:43.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0ec00a21d0 msgr2=0x7f0ec00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0ec00a21d0 0x7f0ec00a4690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc000c00 tx=0x7f0ecc005fd0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc06a2c0 msgr2=0x7f0edc19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:43.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc06a2c0 0x7f0edc19a8f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800e9f0 tx=0x7f0ec800ed00 comp rx=0 tx=0).stop 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 shutdown_connections 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f0ec00a21d0 0x7f0ec00a4690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0edc107cc0 0x7f0edc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0edc069920 0x7f0edc19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 --2- 172.21.15.6:0/604175994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0edc06a2c0 0x7f0edc19a8f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.353+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 >> 172.21.15.6:0/604175994 conn(0x7f0edc077aa0 msgr2=0x7f0edc10ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.354+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 shutdown_connections 2024-02-26T06:26:43.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:43.354+0000 7f0ee1c00700 1 -- 172.21.15.6:0/604175994 wait complete. 2024-02-26T06:26:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:43 smithi006 ceph-mon[31790]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:26:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:43 smithi006 ceph-mon[31790]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-02-26T06:26:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:43 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2147424498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T06:26:44.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:43 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/604175994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T06:26:44.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:43 smithi167 ceph-mon[36229]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:26:44.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:43 smithi167 ceph-mon[36229]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-02-26T06:26:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:43 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2147424498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T06:26:44.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:43 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/604175994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T06:26:44.352 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070512 for osd.8 2024-02-26T06:26:44.352 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:43 smithi090 ceph-mon[35681]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:26:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:43 smithi090 ceph-mon[35681]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-02-26T06:26:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:43 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2147424498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T06:26:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:43 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/604175994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T06:26:45.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757308 got 163208757314 for osd.5 2024-02-26T06:26:45.009 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:45.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495675 for osd.6 2024-02-26T06:26:45.671 DEBUG:teuthology.parallel:result is None 2024-02-26T06:26:45.671 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-26T06:26:45.671 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T06:26:46.185 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:46.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:45 smithi167 ceph-mon[36229]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:46.334 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:45 smithi006 ceph-mon[31790]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:45 smithi090 ceph-mon[35681]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:48.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:47 smithi167 ceph-mon[36229]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-02-26T06:26:48.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:48.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:47 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:48.279 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:47 smithi006 ceph-mon[31790]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-02-26T06:26:48.280 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:48.280 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:47 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:47 smithi090 ceph-mon[35681]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-02-26T06:26:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:47 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:48.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 -- 172.21.15.6:0/721896857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb40747c0 msgr2=0x7f9cb410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 --2- 172.21.15.6:0/721896857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb40747c0 0x7f9cb410e130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00098f0 tx=0x7f9ca0009c00 comp rx=0 tx=0).stop 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 -- 172.21.15.6:0/721896857 shutdown_connections 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 --2- 172.21.15.6:0/721896857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb410e670 0x7f9cb4110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 --2- 172.21.15.6:0/721896857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb4073ea0 0x7f9cb4074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 --2- 172.21.15.6:0/721896857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb40747c0 0x7f9cb410e130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 -- 172.21.15.6:0/721896857 >> 172.21.15.6:0/721896857 conn(0x7f9cb4078dd0 msgr2=0x7f9cb40791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 -- 172.21.15.6:0/721896857 shutdown_connections 2024-02-26T06:26:48.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.450+0000 7f9cbc200700 1 -- 172.21.15.6:0/721896857 wait complete. 2024-02-26T06:26:48.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.451+0000 7f9cbc200700 1 Processor -- start 2024-02-26T06:26:48.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.451+0000 7f9cbc200700 1 -- start start 2024-02-26T06:26:48.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cbc200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb4073ea0 0x7f9cb419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:48.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cbc200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb40747c0 0x7f9cb419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:48.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cbc200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb410e670 0x7f9cb419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:48.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cbc200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb419f360 con 0x7f9cb40747c0 2024-02-26T06:26:48.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cbc200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb419f4d0 con 0x7f9cb4073ea0 2024-02-26T06:26:48.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cbc200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb419f690 con 0x7f9cb410e670 2024-02-26T06:26:48.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cb8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb40747c0 0x7f9cb419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:48.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cb9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb4073ea0 0x7f9cb419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:48.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cb8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb40747c0 0x7f9cb419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45558/0 (socket says 172.21.15.6:45558) 2024-02-26T06:26:48.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cb8e00700 1 -- 172.21.15.6:0/1532994491 learned_addr learned my addr 172.21.15.6:0/1532994491 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:48.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.452+0000 7f9cba200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb410e670 0x7f9cb419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:48.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb9800700 1 -- 172.21.15.6:0/1532994491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb410e670 msgr2=0x7f9cb419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:48.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb9800700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb410e670 0x7f9cb419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb9800700 1 -- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb40747c0 msgr2=0x7f9cb419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:48.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb9800700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb40747c0 0x7f9cb419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb9800700 1 -- 172.21.15.6:0/1532994491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca0009520 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cba200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb410e670 0x7f9cb419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb9800700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb4073ea0 0x7f9cb419a480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800e9f0 tx=0x7f9ca800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80096d0 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb419f970 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ca800cbd0 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8018680 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.453+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb41a71d0 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.455+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb4109880 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.456+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9ca800cd40 con 0x7f9cb4073ea0 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.457+0000 7f9cb2000700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9c9c0a2150 0x7f9c9c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:48.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.458+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9ca8014070 con 0x7f9cb4073ea0 2024-02-26T06:26:48.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.458+0000 7f9cb8e00700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9c9c0a2150 0x7f9c9c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:48.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.459+0000 7f9cb8e00700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9c9c0a2150 0x7f9c9c0a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9cb41017b0 tx=0x7f9ca0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:48.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.462+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ca8052d20 con 0x7f9cb4073ea0 2024-02-26T06:26:48.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.688+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9cb419b7e0 con 0x7f9c9c0a2150 2024-02-26T06:26:48.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.701+0000 7f9cb2000700 1 -- 172.21.15.6:0/1532994491 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342913 (secure 0 0 0) 0x7f9cb419b7e0 con 0x7f9c9c0a2150 2024-02-26T06:26:48.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:26:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9c9c0a2150 msgr2=0x7f9c9c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9c9c0a2150 0x7f9c9c0a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9cb41017b0 tx=0x7f9ca0019040 comp rx=0 tx=0).stop 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb4073ea0 msgr2=0x7f9cb419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb4073ea0 0x7f9cb419a480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800e9f0 tx=0x7f9ca800ed00 comp rx=0 tx=0).stop 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 shutdown_connections 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9c9c0a2150 0x7f9c9c0a4610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb410e670 0x7f9cb419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cb4073ea0 0x7f9cb419a480 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 --2- 172.21.15.6:0/1532994491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb40747c0 0x7f9cb419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 >> 172.21.15.6:0/1532994491 conn(0x7f9cb4078dd0 msgr2=0x7f9cb40fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 shutdown_connections 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:48.708+0000 7f9cbc200700 1 -- 172.21.15.6:0/1532994491 wait complete. 2024-02-26T06:26:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:26:49.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:26:49.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:26:49.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:49.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:26:49.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:26:49.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:49.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:26:49.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:26:49.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:26:49.620 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-02-26T06:26:48.144081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657828,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":979,"num_read_kb":614,"num_write":430,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181457,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383356,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558468,"statfs":{"total":1151940427776,"available":1151547871232,"internally_reserved":0,"allocated":13725696,"data_stored":7442388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004633"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125545+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:26:42.125545+0000","last_peered":"2024-02-26T06:26:42.125545+0000","last_clean":"2024-02-26T06:26:42.125545+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:26:42.125545+0000","last_undegraded":"2024-02-26T06:26:42.125545+0000","last_fullsized":"2024-02-26T06:26:42.125545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121536+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:26:42.121536+0000","last_peered":"2024-02-26T06:26:42.121536+0000","last_clean":"2024-02-26T06:26:42.121536+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:26:42.121536+0000","last_undegraded":"2024-02-26T06:26:42.121536+0000","last_fullsized":"2024-02-26T06:26:42.121536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128147+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:26:42.128147+0000","last_peered":"2024-02-26T06:26:42.128147+0000","last_clean":"2024-02-26T06:26:42.128147+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:26:42.128147+0000","last_undegraded":"2024-02-26T06:26:42.128147+0000","last_fullsized":"2024-02-26T06:26:42.128147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106826+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:26:42.106826+0000","last_peered":"2024-02-26T06:26:42.106826+0000","last_clean":"2024-02-26T06:26:42.106826+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:26:42.106826+0000","last_undegraded":"2024-02-26T06:26:42.106826+0000","last_fullsized":"2024-02-26T06:26:42.106826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101875+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:26:42.101875+0000","last_peered":"2024-02-26T06:26:42.101875+0000","last_clean":"2024-02-26T06:26:42.101875+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:26:42.101875+0000","last_undegraded":"2024-02-26T06:26:42.101875+0000","last_fullsized":"2024-02-26T06:26:42.101875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105400+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:26:42.105400+0000","last_peered":"2024-02-26T06:26:42.105400+0000","last_clean":"2024-02-26T06:26:42.105400+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:26:42.105400+0000","last_undegraded":"2024-02-26T06:26:42.105400+0000","last_fullsized":"2024-02-26T06:26:42.105400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114163+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:26:42.114163+0000","last_peered":"2024-02-26T06:26:42.114163+0000","last_clean":"2024-02-26T06:26:42.114163+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:26:42.114163+0000","last_undegraded":"2024-02-26T06:26:42.114163+0000","last_fullsized":"2024-02-26T06:26:42.114163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123188+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:26:42.123188+0000","last_peered":"2024-02-26T06:26:42.123188+0000","last_clean":"2024-02-26T06:26:42.123188+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:26:42.123188+0000","last_undegraded":"2024-02-26T06:26:42.123188+0000","last_fullsized":"2024-02-26T06:26:42.123188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113809+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:26:42.113809+0000","last_peered":"2024-02-26T06:26:42.113809+0000","last_clean":"2024-02-26T06:26:42.113809+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:26:42.113809+0000","last_undegraded":"2024-02-26T06:26:42.113809+0000","last_fullsized":"2024-02-26T06:26:42.113809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.104562+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:26:42.104562+0000","last_peered":"2024-02-26T06:26:42.104562+0000","last_clean":"2024-02-26T06:26:42.104562+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:26:42.104562+0000","last_undegraded":"2024-02-26T06:26:42.104562+0000","last_fullsized":"2024-02-26T06:26:42.104562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107477+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:26:42.107477+0000","last_peered":"2024-02-26T06:26:42.107477+0000","last_clean":"2024-02-26T06:26:42.107477+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:26:42.107477+0000","last_undegraded":"2024-02-26T06:26:42.107477+0000","last_fullsized":"2024-02-26T06:26:42.107477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100407+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:26:42.100407+0000","last_peered":"2024-02-26T06:26:42.100407+0000","last_clean":"2024-02-26T06:26:42.100407+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:26:42.100407+0000","last_undegraded":"2024-02-26T06:26:42.100407+0000","last_fullsized":"2024-02-26T06:26:42.100407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123361+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:26:42.123361+0000","last_peered":"2024-02-26T06:26:42.123361+0000","last_clean":"2024-02-26T06:26:42.123361+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:26:42.123361+0000","last_undegraded":"2024-02-26T06:26:42.123361+0000","last_fullsized":"2024-02-26T06:26:42.123361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127617+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:26:42.127617+0000","last_peered":"2024-02-26T06:26:42.127617+0000","last_clean":"2024-02-26T06:26:42.127617+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:26:42.127617+0000","last_undegraded":"2024-02-26T06:26:42.127617+0000","last_fullsized":"2024-02-26T06:26:42.127617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124207+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:26:42.124207+0000","last_peered":"2024-02-26T06:26:42.124207+0000","last_clean":"2024-02-26T06:26:42.124207+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:26:42.124207+0000","last_undegraded":"2024-02-26T06:26:42.124207+0000","last_fullsized":"2024-02-26T06:26:42.124207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129267+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:26:42.129267+0000","last_peered":"2024-02-26T06:26:42.129267+0000","last_clean":"2024-02-26T06:26:42.129267+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:26:42.129267+0000","last_undegraded":"2024-02-26T06:26:42.129267+0000","last_fullsized":"2024-02-26T06:26:42.129267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125702+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:26:42.125702+0000","last_peered":"2024-02-26T06:26:42.125702+0000","last_clean":"2024-02-26T06:26:42.125702+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:26:42.125702+0000","last_undegraded":"2024-02-26T06:26:42.125702+0000","last_fullsized":"2024-02-26T06:26:42.125702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107587+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:26:42.107587+0000","last_peered":"2024-02-26T06:26:42.107587+0000","last_clean":"2024-02-26T06:26:42.107587+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:26:42.107587+0000","last_undegraded":"2024-02-26T06:26:42.107587+0000","last_fullsized":"2024-02-26T06:26:42.107587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125040+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:26:42.125040+0000","last_peered":"2024-02-26T06:26:42.125040+0000","last_clean":"2024-02-26T06:26:42.125040+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:26:42.125040+0000","last_undegraded":"2024-02-26T06:26:42.125040+0000","last_fullsized":"2024-02-26T06:26:42.125040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:43.972995+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:26:43.972995+0000","last_peered":"2024-02-26T06:26:43.972995+0000","last_clean":"2024-02-26T06:26:43.972995+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:26:43.972995+0000","last_undegraded":"2024-02-26T06:26:43.972995+0000","last_fullsized":"2024-02-26T06:26:43.972995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120727+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:26:42.120727+0000","last_peered":"2024-02-26T06:26:42.120727+0000","last_clean":"2024-02-26T06:26:42.120727+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:26:42.120727+0000","last_undegraded":"2024-02-26T06:26:42.120727+0000","last_fullsized":"2024-02-26T06:26:42.120727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869401+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:26:13.869401+0000","last_peered":"2024-02-26T06:26:13.869401+0000","last_clean":"2024-02-26T06:26:13.869401+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:26:13.869401+0000","last_undegraded":"2024-02-26T06:26:13.869401+0000","last_fullsized":"2024-02-26T06:26:13.869401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128493+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:26:42.128493+0000","last_peered":"2024-02-26T06:26:42.128493+0000","last_clean":"2024-02-26T06:26:42.128493+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:26:42.128493+0000","last_undegraded":"2024-02-26T06:26:42.128493+0000","last_fullsized":"2024-02-26T06:26:42.128493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.864219+0000","last_change":"2024-02-26T06:26:13.864219+0000","last_active":"2024-02-26T06:26:13.864219+0000","last_peered":"2024-02-26T06:26:13.864219+0000","last_clean":"2024-02-26T06:26:13.864219+0000","last_became_active":"2024-02-26T06:26:13.851415+0000","last_became_peered":"2024-02-26T06:26:13.851415+0000","last_unstale":"2024-02-26T06:26:13.864219+0000","last_undegraded":"2024-02-26T06:26:13.864219+0000","last_fullsized":"2024-02-26T06:26:13.864219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:46:36.684794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125570+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:26:42.125570+0000","last_peered":"2024-02-26T06:26:42.125570+0000","last_clean":"2024-02-26T06:26:42.125570+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:26:42.125570+0000","last_undegraded":"2024-02-26T06:26:42.125570+0000","last_fullsized":"2024-02-26T06:26:42.125570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.874338+0000","last_change":"2024-02-26T06:26:13.873833+0000","last_active":"2024-02-26T06:26:13.874338+0000","last_peered":"2024-02-26T06:26:13.874338+0000","last_clean":"2024-02-26T06:26:13.874338+0000","last_became_active":"2024-02-26T06:26:13.852310+0000","last_became_peered":"2024-02-26T06:26:13.852310+0000","last_unstale":"2024-02-26T06:26:13.874338+0000","last_undegraded":"2024-02-26T06:26:13.874338+0000","last_fullsized":"2024-02-26T06:26:13.874338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:20.887692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.117029+0000","last_change":"2024-02-26T06:26:13.864818+0000","last_active":"2024-02-26T06:26:42.117029+0000","last_peered":"2024-02-26T06:26:42.117029+0000","last_clean":"2024-02-26T06:26:42.117029+0000","last_became_active":"2024-02-26T06:26:13.863821+0000","last_became_peered":"2024-02-26T06:26:13.863821+0000","last_unstale":"2024-02-26T06:26:42.117029+0000","last_undegraded":"2024-02-26T06:26:42.117029+0000","last_fullsized":"2024-02-26T06:26:42.117029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:38.266114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":106,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.909451+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:26:42.909451+0000","last_peered":"2024-02-26T06:26:42.909451+0000","last_clean":"2024-02-26T06:26:42.909451+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:26:42.909451+0000","last_undegraded":"2024-02-26T06:26:42.909451+0000","last_fullsized":"2024-02-26T06:26:42.909451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098741+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:26:42.098741+0000","last_peered":"2024-02-26T06:26:42.098741+0000","last_clean":"2024-02-26T06:26:42.098741+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:26:42.098741+0000","last_undegraded":"2024-02-26T06:26:42.098741+0000","last_fullsized":"2024-02-26T06:26:42.098741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851699+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:26:13.851699+0000","last_peered":"2024-02-26T06:26:13.851699+0000","last_clean":"2024-02-26T06:26:13.851699+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:26:13.851699+0000","last_undegraded":"2024-02-26T06:26:13.851699+0000","last_fullsized":"2024-02-26T06:26:13.851699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131059+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:26:42.131059+0000","last_peered":"2024-02-26T06:26:42.131059+0000","last_clean":"2024-02-26T06:26:42.131059+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:26:42.131059+0000","last_undegraded":"2024-02-26T06:26:42.131059+0000","last_fullsized":"2024-02-26T06:26:42.131059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.858204+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:26:13.858204+0000","last_peered":"2024-02-26T06:26:13.858204+0000","last_clean":"2024-02-26T06:26:13.858204+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:26:13.858204+0000","last_undegraded":"2024-02-26T06:26:13.858204+0000","last_fullsized":"2024-02-26T06:26:13.858204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128016+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:26:42.128016+0000","last_peered":"2024-02-26T06:26:42.128016+0000","last_clean":"2024-02-26T06:26:42.128016+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:26:42.128016+0000","last_undegraded":"2024-02-26T06:26:42.128016+0000","last_fullsized":"2024-02-26T06:26:42.128016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.850718+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:26:13.850718+0000","last_peered":"2024-02-26T06:26:13.850718+0000","last_clean":"2024-02-26T06:26:13.850718+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:26:13.850718+0000","last_undegraded":"2024-02-26T06:26:13.850718+0000","last_fullsized":"2024-02-26T06:26:13.850718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131552+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:26:42.131552+0000","last_peered":"2024-02-26T06:26:42.131552+0000","last_clean":"2024-02-26T06:26:42.131552+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:26:42.131552+0000","last_undegraded":"2024-02-26T06:26:42.131552+0000","last_fullsized":"2024-02-26T06:26:42.131552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":183,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:43.910064+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:26:43.910064+0000","last_peered":"2024-02-26T06:26:43.910064+0000","last_clean":"2024-02-26T06:26:43.910064+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:26:43.910064+0000","last_undegraded":"2024-02-26T06:26:43.910064+0000","last_fullsized":"2024-02-26T06:26:43.910064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":160,"num_read_kb":160,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.115681+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:26:42.115681+0000","last_peered":"2024-02-26T06:26:42.115681+0000","last_clean":"2024-02-26T06:26:42.115681+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:26:42.115681+0000","last_undegraded":"2024-02-26T06:26:42.115681+0000","last_fullsized":"2024-02-26T06:26:42.115681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848244+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:26:13.848244+0000","last_peered":"2024-02-26T06:26:13.848244+0000","last_clean":"2024-02-26T06:26:13.848244+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:26:13.848244+0000","last_undegraded":"2024-02-26T06:26:13.848244+0000","last_fullsized":"2024-02-26T06:26:13.848244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128168+0000","last_change":"2024-02-26T06:26:13.858594+0000","last_active":"2024-02-26T06:26:42.128168+0000","last_peered":"2024-02-26T06:26:42.128168+0000","last_clean":"2024-02-26T06:26:42.128168+0000","last_became_active":"2024-02-26T06:26:13.852999+0000","last_became_peered":"2024-02-26T06:26:13.852999+0000","last_unstale":"2024-02-26T06:26:42.128168+0000","last_undegraded":"2024-02-26T06:26:42.128168+0000","last_fullsized":"2024-02-26T06:26:42.128168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:17:51.940824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851762+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:26:13.851762+0000","last_peered":"2024-02-26T06:26:13.851762+0000","last_clean":"2024-02-26T06:26:13.851762+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:26:13.851762+0000","last_undegraded":"2024-02-26T06:26:13.851762+0000","last_fullsized":"2024-02-26T06:26:13.851762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125747+0000","last_change":"2024-02-26T06:26:13.863050+0000","last_active":"2024-02-26T06:26:42.125747+0000","last_peered":"2024-02-26T06:26:42.125747+0000","last_clean":"2024-02-26T06:26:42.125747+0000","last_became_active":"2024-02-26T06:26:13.862635+0000","last_became_peered":"2024-02-26T06:26:13.862635+0000","last_unstale":"2024-02-26T06:26:42.125747+0000","last_undegraded":"2024-02-26T06:26:42.125747+0000","last_fullsized":"2024-02-26T06:26:42.125747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:38:45.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":223,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:43.910940+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:26:43.910940+0000","last_peered":"2024-02-26T06:26:43.910940+0000","last_clean":"2024-02-26T06:26:43.910940+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:26:43.910940+0000","last_undegraded":"2024-02-26T06:26:43.910940+0000","last_fullsized":"2024-02-26T06:26:43.910940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":169,"num_read_kb":144,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099872+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:26:42.099872+0000","last_peered":"2024-02-26T06:26:42.099872+0000","last_clean":"2024-02-26T06:26:42.099872+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:26:42.099872+0000","last_undegraded":"2024-02-26T06:26:42.099872+0000","last_fullsized":"2024-02-26T06:26:42.099872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848072+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:26:13.848072+0000","last_peered":"2024-02-26T06:26:13.848072+0000","last_clean":"2024-02-26T06:26:13.848072+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:26:13.848072+0000","last_undegraded":"2024-02-26T06:26:13.848072+0000","last_fullsized":"2024-02-26T06:26:13.848072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131700+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:26:42.131700+0000","last_peered":"2024-02-26T06:26:42.131700+0000","last_clean":"2024-02-26T06:26:42.131700+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:26:42.131700+0000","last_undegraded":"2024-02-26T06:26:42.131700+0000","last_fullsized":"2024-02-26T06:26:42.131700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851099+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:26:13.851099+0000","last_peered":"2024-02-26T06:26:13.851099+0000","last_clean":"2024-02-26T06:26:13.851099+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:26:13.851099+0000","last_undegraded":"2024-02-26T06:26:13.851099+0000","last_fullsized":"2024-02-26T06:26:13.851099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112214+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:26:42.112214+0000","last_peered":"2024-02-26T06:26:42.112214+0000","last_clean":"2024-02-26T06:26:42.112214+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:26:42.112214+0000","last_undegraded":"2024-02-26T06:26:42.112214+0000","last_fullsized":"2024-02-26T06:26:42.112214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.974371+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:26:42.974371+0000","last_peered":"2024-02-26T06:26:42.974371+0000","last_clean":"2024-02-26T06:26:42.974371+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:26:42.974371+0000","last_undegraded":"2024-02-26T06:26:42.974371+0000","last_fullsized":"2024-02-26T06:26:42.974371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112955+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:26:42.112955+0000","last_peered":"2024-02-26T06:26:42.112955+0000","last_clean":"2024-02-26T06:26:42.112955+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:26:42.112955+0000","last_undegraded":"2024-02-26T06:26:42.112955+0000","last_fullsized":"2024-02-26T06:26:42.112955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844664+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:26:13.844664+0000","last_peered":"2024-02-26T06:26:13.844664+0000","last_clean":"2024-02-26T06:26:13.844664+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:26:13.844664+0000","last_undegraded":"2024-02-26T06:26:13.844664+0000","last_fullsized":"2024-02-26T06:26:13.844664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113707+0000","last_change":"2024-02-26T06:26:13.862455+0000","last_active":"2024-02-26T06:26:42.113707+0000","last_peered":"2024-02-26T06:26:42.113707+0000","last_clean":"2024-02-26T06:26:42.113707+0000","last_became_active":"2024-02-26T06:26:13.861385+0000","last_became_peered":"2024-02-26T06:26:13.861385+0000","last_unstale":"2024-02-26T06:26:42.113707+0000","last_undegraded":"2024-02-26T06:26:42.113707+0000","last_fullsized":"2024-02-26T06:26:42.113707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:42:41.345104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869092+0000","last_change":"2024-02-26T06:26:13.868540+0000","last_active":"2024-02-26T06:26:13.869092+0000","last_peered":"2024-02-26T06:26:13.869092+0000","last_clean":"2024-02-26T06:26:13.869092+0000","last_became_active":"2024-02-26T06:26:13.852149+0000","last_became_peered":"2024-02-26T06:26:13.852149+0000","last_unstale":"2024-02-26T06:26:13.869092+0000","last_undegraded":"2024-02-26T06:26:13.869092+0000","last_fullsized":"2024-02-26T06:26:13.869092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:14:30.745782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101245+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:26:42.101245+0000","last_peered":"2024-02-26T06:26:42.101245+0000","last_clean":"2024-02-26T06:26:42.101245+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:26:42.101245+0000","last_undegraded":"2024-02-26T06:26:42.101245+0000","last_fullsized":"2024-02-26T06:26:42.101245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.859894+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:26:13.859894+0000","last_peered":"2024-02-26T06:26:13.859894+0000","last_clean":"2024-02-26T06:26:13.859894+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:26:13.859894+0000","last_undegraded":"2024-02-26T06:26:13.859894+0000","last_fullsized":"2024-02-26T06:26:13.859894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.103414+0000","last_change":"2024-02-26T06:26:13.872244+0000","last_active":"2024-02-26T06:26:42.103414+0000","last_peered":"2024-02-26T06:26:42.103414+0000","last_clean":"2024-02-26T06:26:42.103414+0000","last_became_active":"2024-02-26T06:26:13.866094+0000","last_became_peered":"2024-02-26T06:26:13.866094+0000","last_unstale":"2024-02-26T06:26:42.103414+0000","last_undegraded":"2024-02-26T06:26:42.103414+0000","last_fullsized":"2024-02-26T06:26:42.103414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:56:52.301479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":106,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:43.974056+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:26:43.974056+0000","last_peered":"2024-02-26T06:26:43.974056+0000","last_clean":"2024-02-26T06:26:43.974056+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:26:43.974056+0000","last_undegraded":"2024-02-26T06:26:43.974056+0000","last_fullsized":"2024-02-26T06:26:43.974056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.102768+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:26:42.102768+0000","last_peered":"2024-02-26T06:26:42.102768+0000","last_clean":"2024-02-26T06:26:42.102768+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:26:42.102768+0000","last_undegraded":"2024-02-26T06:26:42.102768+0000","last_fullsized":"2024-02-26T06:26:42.102768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.093903+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:26:42.093903+0000","last_peered":"2024-02-26T06:26:42.093903+0000","last_clean":"2024-02-26T06:26:42.093903+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:26:42.093903+0000","last_undegraded":"2024-02-26T06:26:42.093903+0000","last_fullsized":"2024-02-26T06:26:42.093903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:43.975630+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:26:43.975630+0000","last_peered":"2024-02-26T06:26:43.975630+0000","last_clean":"2024-02-26T06:26:43.975630+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:26:43.975630+0000","last_undegraded":"2024-02-26T06:26:43.975630+0000","last_fullsized":"2024-02-26T06:26:43.975630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130301+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:26:42.130301+0000","last_peered":"2024-02-26T06:26:42.130301+0000","last_clean":"2024-02-26T06:26:42.130301+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:26:42.130301+0000","last_undegraded":"2024-02-26T06:26:42.130301+0000","last_fullsized":"2024-02-26T06:26:42.130301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849897+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:26:13.849897+0000","last_peered":"2024-02-26T06:26:13.849897+0000","last_clean":"2024-02-26T06:26:13.849897+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:26:13.849897+0000","last_undegraded":"2024-02-26T06:26:13.849897+0000","last_fullsized":"2024-02-26T06:26:13.849897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846775+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:26:13.846775+0000","last_peered":"2024-02-26T06:26:13.846775+0000","last_clean":"2024-02-26T06:26:13.846775+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:26:13.846775+0000","last_undegraded":"2024-02-26T06:26:13.846775+0000","last_fullsized":"2024-02-26T06:26:13.846775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120068+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:26:42.120068+0000","last_peered":"2024-02-26T06:26:42.120068+0000","last_clean":"2024-02-26T06:26:42.120068+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:26:42.120068+0000","last_undegraded":"2024-02-26T06:26:42.120068+0000","last_fullsized":"2024-02-26T06:26:42.120068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846836+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:26:13.846836+0000","last_peered":"2024-02-26T06:26:13.846836+0000","last_clean":"2024-02-26T06:26:13.846836+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:26:13.846836+0000","last_undegraded":"2024-02-26T06:26:13.846836+0000","last_fullsized":"2024-02-26T06:26:13.846836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.845030+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:26:13.845030+0000","last_peered":"2024-02-26T06:26:13.845030+0000","last_clean":"2024-02-26T06:26:13.845030+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:26:13.845030+0000","last_undegraded":"2024-02-26T06:26:13.845030+0000","last_fullsized":"2024-02-26T06:26:13.845030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.122929+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:26:42.122929+0000","last_peered":"2024-02-26T06:26:42.122929+0000","last_clean":"2024-02-26T06:26:42.122929+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:26:42.122929+0000","last_undegraded":"2024-02-26T06:26:42.122929+0000","last_fullsized":"2024-02-26T06:26:42.122929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'14","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.860053+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:26:13.860053+0000","last_peered":"2024-02-26T06:26:13.860053+0000","last_clean":"2024-02-26T06:26:13.860053+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:26:13.860053+0000","last_undegraded":"2024-02-26T06:26:13.860053+0000","last_fullsized":"2024-02-26T06:26:13.860053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120512+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:26:42.120512+0000","last_peered":"2024-02-26T06:26:42.120512+0000","last_clean":"2024-02-26T06:26:42.120512+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:26:42.120512+0000","last_undegraded":"2024-02-26T06:26:42.120512+0000","last_fullsized":"2024-02-26T06:26:42.120512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849919+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:26:13.849919+0000","last_peered":"2024-02-26T06:26:13.849919+0000","last_clean":"2024-02-26T06:26:13.849919+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:26:13.849919+0000","last_undegraded":"2024-02-26T06:26:13.849919+0000","last_fullsized":"2024-02-26T06:26:13.849919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128451+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:26:42.128451+0000","last_peered":"2024-02-26T06:26:42.128451+0000","last_clean":"2024-02-26T06:26:42.128451+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:26:42.128451+0000","last_undegraded":"2024-02-26T06:26:42.128451+0000","last_fullsized":"2024-02-26T06:26:42.128451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.847060+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:26:13.847060+0000","last_peered":"2024-02-26T06:26:13.847060+0000","last_clean":"2024-02-26T06:26:13.847060+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:26:13.847060+0000","last_undegraded":"2024-02-26T06:26:13.847060+0000","last_fullsized":"2024-02-26T06:26:13.847060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.069585+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:26:42.069585+0000","last_peered":"2024-02-26T06:26:42.069585+0000","last_clean":"2024-02-26T06:26:42.069585+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:26:42.069585+0000","last_undegraded":"2024-02-26T06:26:42.069585+0000","last_fullsized":"2024-02-26T06:26:42.069585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.866190+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:26:13.866190+0000","last_peered":"2024-02-26T06:26:13.866190+0000","last_clean":"2024-02-26T06:26:13.866190+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:26:13.866190+0000","last_undegraded":"2024-02-26T06:26:13.866190+0000","last_fullsized":"2024-02-26T06:26:13.866190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105913+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:26:42.105913+0000","last_peered":"2024-02-26T06:26:42.105913+0000","last_clean":"2024-02-26T06:26:42.105913+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:26:42.105913+0000","last_undegraded":"2024-02-26T06:26:42.105913+0000","last_fullsized":"2024-02-26T06:26:42.105913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848263+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:26:13.848263+0000","last_peered":"2024-02-26T06:26:13.848263+0000","last_clean":"2024-02-26T06:26:13.848263+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:26:13.848263+0000","last_undegraded":"2024-02-26T06:26:13.848263+0000","last_fullsized":"2024-02-26T06:26:13.848263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129595+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:26:42.129595+0000","last_peered":"2024-02-26T06:26:42.129595+0000","last_clean":"2024-02-26T06:26:42.129595+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:26:42.129595+0000","last_undegraded":"2024-02-26T06:26:42.129595+0000","last_fullsized":"2024-02-26T06:26:42.129595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844759+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:26:13.844759+0000","last_peered":"2024-02-26T06:26:13.844759+0000","last_clean":"2024-02-26T06:26:13.844759+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:26:13.844759+0000","last_undegraded":"2024-02-26T06:26:13.844759+0000","last_fullsized":"2024-02-26T06:26:13.844759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.111151+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:26:42.111151+0000","last_peered":"2024-02-26T06:26:42.111151+0000","last_clean":"2024-02-26T06:26:42.111151+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:26:42.111151+0000","last_undegraded":"2024-02-26T06:26:42.111151+0000","last_fullsized":"2024-02-26T06:26:42.111151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:44.912205+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:26:44.912205+0000","last_peered":"2024-02-26T06:26:44.912205+0000","last_clean":"2024-02-26T06:26:44.912205+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:26:44.912205+0000","last_undegraded":"2024-02-26T06:26:44.912205+0000","last_fullsized":"2024-02-26T06:26:44.912205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107584+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:26:42.107584+0000","last_peered":"2024-02-26T06:26:42.107584+0000","last_clean":"2024-02-26T06:26:42.107584+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:26:42.107584+0000","last_undegraded":"2024-02-26T06:26:42.107584+0000","last_fullsized":"2024-02-26T06:26:42.107584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851618+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:26:13.851618+0000","last_peered":"2024-02-26T06:26:13.851618+0000","last_clean":"2024-02-26T06:26:13.851618+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:26:13.851618+0000","last_undegraded":"2024-02-26T06:26:13.851618+0000","last_fullsized":"2024-02-26T06:26:13.851618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106886+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:26:42.106886+0000","last_peered":"2024-02-26T06:26:42.106886+0000","last_clean":"2024-02-26T06:26:42.106886+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:26:42.106886+0000","last_undegraded":"2024-02-26T06:26:42.106886+0000","last_fullsized":"2024-02-26T06:26:42.106886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.878533+0000","last_change":"2024-02-26T06:26:13.878533+0000","last_active":"2024-02-26T06:26:13.878533+0000","last_peered":"2024-02-26T06:26:13.878533+0000","last_clean":"2024-02-26T06:26:13.878533+0000","last_became_active":"2024-02-26T06:26:13.866478+0000","last_became_peered":"2024-02-26T06:26:13.866478+0000","last_unstale":"2024-02-26T06:26:13.878533+0000","last_undegraded":"2024-02-26T06:26:13.878533+0000","last_fullsized":"2024-02-26T06:26:13.878533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:00:23.896436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099199+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:26:42.099199+0000","last_peered":"2024-02-26T06:26:42.099199+0000","last_clean":"2024-02-26T06:26:42.099199+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:26:42.099199+0000","last_undegraded":"2024-02-26T06:26:42.099199+0000","last_fullsized":"2024-02-26T06:26:42.099199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099772+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:26:42.099772+0000","last_peered":"2024-02-26T06:26:42.099772+0000","last_clean":"2024-02-26T06:26:42.099772+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:26:42.099772+0000","last_undegraded":"2024-02-26T06:26:42.099772+0000","last_fullsized":"2024-02-26T06:26:42.099772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.108107+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:26:42.108107+0000","last_peered":"2024-02-26T06:26:42.108107+0000","last_clean":"2024-02-26T06:26:42.108107+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:26:42.108107+0000","last_undegraded":"2024-02-26T06:26:42.108107+0000","last_fullsized":"2024-02-26T06:26:42.108107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123861+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:26:42.123861+0000","last_peered":"2024-02-26T06:26:42.123861+0000","last_clean":"2024-02-26T06:26:42.123861+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:26:42.123861+0000","last_undegraded":"2024-02-26T06:26:42.123861+0000","last_fullsized":"2024-02-26T06:26:42.123861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127224+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:26:42.127224+0000","last_peered":"2024-02-26T06:26:42.127224+0000","last_clean":"2024-02-26T06:26:42.127224+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:26:42.127224+0000","last_undegraded":"2024-02-26T06:26:42.127224+0000","last_fullsized":"2024-02-26T06:26:42.127224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126738+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:26:42.126738+0000","last_peered":"2024-02-26T06:26:42.126738+0000","last_clean":"2024-02-26T06:26:42.126738+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:26:42.126738+0000","last_undegraded":"2024-02-26T06:26:42.126738+0000","last_fullsized":"2024-02-26T06:26:42.126738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.096657+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:26:42.096657+0000","last_peered":"2024-02-26T06:26:42.096657+0000","last_clean":"2024-02-26T06:26:42.096657+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:26:42.096657+0000","last_undegraded":"2024-02-26T06:26:42.096657+0000","last_fullsized":"2024-02-26T06:26:42.096657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106173+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:26:42.106173+0000","last_peered":"2024-02-26T06:26:42.106173+0000","last_clean":"2024-02-26T06:26:42.106173+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:26:42.106173+0000","last_undegraded":"2024-02-26T06:26:42.106173+0000","last_fullsized":"2024-02-26T06:26:42.106173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112171+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:26:42.112171+0000","last_peered":"2024-02-26T06:26:42.112171+0000","last_clean":"2024-02-26T06:26:42.112171+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:26:42.112171+0000","last_undegraded":"2024-02-26T06:26:42.112171+0000","last_fullsized":"2024-02-26T06:26:42.112171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130606+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:26:42.130606+0000","last_peered":"2024-02-26T06:26:42.130606+0000","last_clean":"2024-02-26T06:26:42.130606+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:26:42.130606+0000","last_undegraded":"2024-02-26T06:26:42.130606+0000","last_fullsized":"2024-02-26T06:26:42.130606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120820+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:26:42.120820+0000","last_peered":"2024-02-26T06:26:42.120820+0000","last_clean":"2024-02-26T06:26:42.120820+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:26:42.120820+0000","last_undegraded":"2024-02-26T06:26:42.120820+0000","last_fullsized":"2024-02-26T06:26:42.120820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131628+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:26:42.131628+0000","last_peered":"2024-02-26T06:26:42.131628+0000","last_clean":"2024-02-26T06:26:42.131628+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:26:42.131628+0000","last_undegraded":"2024-02-26T06:26:42.131628+0000","last_fullsized":"2024-02-26T06:26:42.131628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129177+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:26:42.129177+0000","last_peered":"2024-02-26T06:26:42.129177+0000","last_clean":"2024-02-26T06:26:42.129177+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:26:42.129177+0000","last_undegraded":"2024-02-26T06:26:42.129177+0000","last_fullsized":"2024-02-26T06:26:42.129177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100619+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:26:42.100619+0000","last_peered":"2024-02-26T06:26:42.100619+0000","last_clean":"2024-02-26T06:26:42.100619+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:26:42.100619+0000","last_undegraded":"2024-02-26T06:26:42.100619+0000","last_fullsized":"2024-02-26T06:26:42.100619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67226,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":843,"num_read_kb":495,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.621}]},{"osd":1,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.33500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63500000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.5},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.874,"15min":0.874},"last":0.41799999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0680000000000001,"5min":1.103,"15min":1.103},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3779999999999999,"5min":1.385,"15min":1.385},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.58999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.90100000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.65300000000000002},"min":{"1min":0.53000000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.79700000000000004}]},{"osd":8,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52900000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.167},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0229999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71099999999999997,"15min":0.69999999999999996},"min":{"1min":0.61099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.044,"15min":1.044},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.63600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.97799999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.78300000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1108767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.7549999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.34000000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82099999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88700000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.117,"5min":1.458,"15min":1.458},"last":0.78700000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.70599999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.38200000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.72199999999999998,"15min":0.73899999999999999},"min":{"1min":0.497,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80300000000000005,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.84999999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.85399999999999998,"15min":0.86599999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.069,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.145,"15min":1.145},"last":1.002}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73799999999999999,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.82499999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84999999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64000000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1830000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.97299999999999998,"15min":0.99099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.256},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Feb 26 06:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.621},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93000000000000005,"5min":1.014,"15min":1.014},"last":0.58599999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.4079999999999999,"15min":1.4330000000000001},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":2.1709999999999998,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.444,"15min":1.454},"min":{"1min":0.72099999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9239999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.1180000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1419999999999999,"15min":1.1890000000000001},"min":{"1min":0.75900000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.621,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.173},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.306},"min":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7450000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.393}]},{"osd":2,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.5089999999999999,"15min":1.5569999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.1819999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.359,"15min":1.401},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.919,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.7569999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.681,"15min":1.7569999999999999},"min":{"1min":1.0169999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.601,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.367},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.8100000000000001,"15min":1.8620000000000001},"min":{"1min":0.79400000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.242,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":2.1619999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.694,"15min":1.772},"min":{"1min":0.94199999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.883},{"interface":"front","average":{"1min":1.639,"5min":1.8540000000000001,"15min":1.9450000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.4409999999999998,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":2.0779999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.736,"5min":2.0779999999999998,"15min":2.0920000000000001},"min":{"1min":0.92300000000000004,"5min":0.879,"15min":0.879},"max":{"1min":2.4460000000000002,"5min":14.581,"15min":14.581},"last":2.298},{"interface":"front","average":{"1min":1.774,"5min":2.5979999999999999,"15min":2.8300000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.867,"5min":18.216000000000001,"15min":18.216000000000001},"last":1.99}]},{"osd":6,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0550000000000002,"5min":2.202,"15min":2.2879999999999998},"min":{"1min":1.175,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.762,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":2.4089999999999998},{"interface":"front","average":{"1min":1.9850000000000001,"5min":2.1269999999999998,"15min":2.2250000000000001},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.98,"5min":3.044,"15min":3.044},"last":2.129}]},{"osd":7,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8420000000000001,"5min":2.012,"15min":2.0880000000000001},"min":{"1min":0.95799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.4289999999999998,"5min":3.036,"15min":3.036},"last":2.2170000000000001},{"interface":"front","average":{"1min":1.972,"5min":2.121,"15min":2.1899999999999999},"min":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.5630000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":2.4870000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":2.2210000000000001,"15min":2.3159999999999998},"min":{"1min":1.147,"5min":1.0469999999999999,"15min":1.0469999999999999},"max":{"1min":2.6659999999999999,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.5409999999999999},{"interface":"front","average":{"1min":1.9970000000000001,"5min":2.0920000000000001,"15min":2.1800000000000002},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":3.097,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":2.0649999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.895,"5min":1.954,"15min":1.964},"min":{"1min":1.026,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.544,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":2.3540000000000001},{"interface":"front","average":{"1min":2.1160000000000001,"5min":2.1899999999999999,"15min":2.202},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":2.5939999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.755}]},{"osd":1,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.73399999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.83299999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.439},"min":{"1min":0.25800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.439,"15min":0.436},"min":{"1min":0.311,"5min":0.224,"15min":0.224},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0720000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.621}]},{"osd":4,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.876,"15min":0.876},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71399999999999997,"5min":0.876,"15min":0.876},"last":0.54900000000000004}]},{"osd":5,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73499999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.43099999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.008,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.90400000000000003,"15min":0.92300000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0309999999999999,"5min":1.373,"15min":1.373},"last":0.85299999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":1.048,"15min":1.048},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60799999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1859999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.97399999999999998,"15min":1.0069999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.165,"5min":1.429,"15min":1.429},"last":0.97399999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88500000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65300000000000002}]},{"osd":11,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56899999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1049901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42299999999999999,"15min":0.40400000000000003},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.44},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.42899999999999999},"min":{"1min":0.36399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.60399999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.98599999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0029999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.68500000000000005}]},{"osd":4,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.5,"15min":0.495},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.625,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61599999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39800000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80500000000000005,"5min":0.877,"15min":0.877},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83999999999999997,"15min":0.84399999999999997},"min":{"1min":0.60699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93899999999999995,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.91700000000000004,"15min":0.872},"min":{"1min":0.62,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":6.9900000000000002,"5min":6.9900000000000002,"15min":6.9900000000000002},"last":0.82399999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.98199999999999998,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0680000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.089,"15min":1.046},"min":{"1min":0.61499999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.0380000000000003,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.90700000000000003}]},{"osd":10,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84299999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69399999999999995}]},{"osd":11,"last update":"Mon Feb 26 06:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.748},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73799999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.206,"15min":1.198},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.262,"5min":1.298,"15min":1.304},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.2080000000000002,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.0229999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0589999999999999,"15min":1.032},"min":{"1min":0.70799999999999996,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":1.105},{"interface":"front","average":{"1min":1.097,"5min":1.1299999999999999,"15min":1.1100000000000001},"min":{"1min":0.78800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.9279999999999999,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.347}]},{"osd":2,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.143},"min":{"1min":0.59399999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6930000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.276,"15min":1.228},"min":{"1min":0.65400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.0209999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.5409999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5469999999999999,"15min":1.5269999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.4780000000000002,"5min":3.113,"15min":3.113},"last":1.994},{"interface":"front","average":{"1min":1.466,"5min":1.629,"15min":1.649},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.258,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.619}]},{"osd":4,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3720000000000001,"15min":1.3560000000000001},"min":{"1min":0.65200000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0009999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.385},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.552,"15min":1.506},"min":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2410000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":2.1429999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.575,"15min":1.617},"min":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.4239999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.762},{"interface":"front","average":{"1min":1.446,"5min":1.607,"15min":1.6000000000000001},"min":{"1min":0.67400000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.3319999999999999,"5min":2.798,"15min":2.798},"last":1.881}]},{"osd":7,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.6759999999999999,"15min":1.5489999999999999},"min":{"1min":0.85799999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.488,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":2.2559999999999998},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.6659999999999999,"15min":1.6120000000000001},"min":{"1min":0.78700000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.5459999999999998,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":2.089}]},{"osd":8,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":1.782,"15min":1.726},"min":{"1min":0.86299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5720000000000001,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":2.3319999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.76,"15min":1.742},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6970000000000001,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":1.9259999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":2.1059999999999999,"15min":2.2559999999999998},"min":{"1min":0.98299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.395,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":2.2120000000000002},{"interface":"front","average":{"1min":1.8839999999999999,"5min":2.2160000000000002,"15min":2.3580000000000001},"min":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.6349999999999998,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":2.3900000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":2.0680000000000001,"15min":2.1280000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.5779999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":2.2869999999999999},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.988,"15min":2.0150000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.5640000000000001,"5min":2.806,"15min":2.806},"last":2.0299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":517766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":0.62},"min":{"1min":0.443,"5min":0.436,"15min":0.374},"max":{"1min":1.0249999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71999999999999997,"15min":0.65500000000000003},"min":{"1min":0.46800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.24,"5min":1.252,"15min":1.252},"last":0.86899999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Feb 26 06:26:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81599999999999995,"15min":0.80100000000000005},"min":{"1min":0.55200000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1639999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.89300000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.34599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.43099999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.621}]},{"osd":6,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0669999999999999,"5min":1.196,"15min":1.196},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70299999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52400000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.52900000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.57599999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90100000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63500000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.754},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.83799999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66500000000000004}]},{"osd":11,"last update":"Mon Feb 26 06:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71499999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":457920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92100000000000004,"15min":0.79800000000000004},"min":{"1min":0.56599999999999995,"5min":0.51500000000000001,"15min":0.45900000000000002},"max":{"1min":1.2869999999999999,"5min":1.569,"15min":1.569},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.81699999999999995},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.40899999999999997},"max":{"1min":1.3959999999999999,"5min":3.516,"15min":3.516},"last":1.196}]},{"osd":2,"last update":"Mon Feb 26 06:26:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.01,"15min":0.90000000000000002},"min":{"1min":0.56499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5049999999999999,"5min":1.667,"15min":1.667},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.1619999999999999,"15min":0.97999999999999998},"min":{"1min":0.495,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.724,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.278},{"interface":"front","average":{"1min":1.798,"5min":1.798,"15min":1.798},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":1.0289999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.413,"15min":1.3340000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.411,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.236},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.52900000000000003,"5min":0.499,"15min":0.499},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.054}]},{"osd":5,"last update":"Mon Feb 26 06:26:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.345,"15min":1.3149999999999999},"min":{"1min":0.71799999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1459999999999999,"5min":2.653,"15min":2.653},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.6619999999999999,"15min":1.54},"min":{"1min":1.028,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.4610000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.488,"15min":1.4379999999999999},"min":{"1min":0.65400000000000003,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4889999999999999,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.631,"5min":1.673,"15min":1.569},"min":{"1min":0.84699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.6619999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.4750000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:25:57 2024","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.698,"15min":1.667},"min":{"1min":0.90900000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3090000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.8660000000000001,"5min":1.843,"15min":1.7989999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.5589999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":2.1080000000000001,"15min":2.1989999999999998},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.774,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.907},{"interface":"front","average":{"1min":2.1059999999999999,"5min":2.2090000000000001,"15min":2.2850000000000001},"min":{"1min":1.071,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.621}]},{"osd":10,"last update":"Mon Feb 26 06:25:57 2024","interfaces":[{"interface":"back","average":{"1min":1.9810000000000001,"5min":2.1219999999999999,"15min":2.1459999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.593},{"interface":"front","average":{"1min":1.494,"5min":1.702,"15min":1.7370000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.3140000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.4399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":1.861,"5min":1.8720000000000001,"15min":1.8740000000000001},"min":{"1min":0.97999999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.681,"15min":1.6850000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3690000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3999999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.40600000000000003},"max":{"1min":0.95999999999999996,"5min":1.163,"15min":1.163},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71899999999999997,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0900000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.38600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.73799999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.35499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69099999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.78400000000000003,"15min":0.71599999999999997},"min":{"1min":0.497,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.087,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73399999999999999,"15min":0.69199999999999995},"min":{"1min":0.57699999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.80300000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.38600000000000001},"min":{"1min":0.36199999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.44500000000000001,"15min":0.42299999999999999},"min":{"1min":0.39300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.67800000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88,"15min":0.85699999999999998},"min":{"1min":0.69599999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.89600000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.754}]},{"osd":9,"last update":"Mon Feb 26 06:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.63200000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.94599999999999995,"15min":0.96199999999999997},"min":{"1min":0.66900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.194,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.54400000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81799999999999995,"5min":0.875,"15min":0.875},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84399999999999997,"5min":1.089,"15min":1.089},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Feb 26 06:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.873},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":456289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.165,"15min":0.96299999999999997},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.867,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.0249999999999999},"min":{"1min":0.45400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0499999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.3420000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0980000000000001,"15min":0.91400000000000003},"min":{"1min":0.78500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.417,"5min":1.157,"15min":0.97999999999999998},"min":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":6.8369999999999997,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2230000000000001,"15min":1.0609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2789999999999999,"15min":1.125},"min":{"1min":0.67800000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2849999999999999,"15min":1.1060000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.0019999999999998,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.421,"15min":1.2470000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.0129999999999999,"5min":3.323,"15min":3.323},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.498,"15min":1.3580000000000001},"min":{"1min":0.91600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.8479999999999999,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":1.272},{"interface":"front","average":{"1min":1.548,"5min":1.5549999999999999,"15min":1.5089999999999999},"min":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.1349999999999998,"5min":5.7770000000000001,"15min":5.7770000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.72,"5min":1.8300000000000001,"15min":1.74},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":3.3159999999999998,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.504},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.643,"15min":1.49},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.1760000000000002,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.762,"15min":1.706},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.544,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.4750000000000001},{"interface":"front","average":{"1min":2.0129999999999999,"5min":1.911,"15min":1.8520000000000001},"min":{"1min":0.82699999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.758,"5min":6.758,"15min":6.758},"last":0.75}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":2.073,"15min":2.1739999999999999},"min":{"1min":0.73499999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":2.0030000000000001,"5min":2.1749999999999998,"15min":2.266},"min":{"1min":1.0820000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.9729999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":1.4099999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":2.1419999999999999,"15min":2.202},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.5049999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.6719999999999999,"5min":2.1160000000000001,"15min":2.1899999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.2349999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.76600000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.573},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0510000000000002,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":0.79600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.312,"5min":0.219,"15min":0.219},"max":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.29499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.627},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":1.04,"15min":1.04},"last":0.77700000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.71199999999999997},"min":{"1min":0.63200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.024,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.76800000000000002,"15min":0.70799999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0620000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60499999999999998,"15min":0.56100000000000005},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81899999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.55900000000000005},"min":{"1min":0.48099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84799999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.82299999999999995,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.036,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98999999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.49199999999999999},"min":{"1min":0.40100000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.875,"15min":0.875},"last":0.94499999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.65500000000000003},"min":{"1min":0.48899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.754},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88,"15min":0.90000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.997,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.054,"5min":1.665,"15min":1.665},"last":0.92600000000000005}]},{"osd":10,"last update":"Mon Feb 26 06:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67300000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81899999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.88},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89300000000000002}]},{"osd":11,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.97199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1049047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0489999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82599999999999996,"15min":0.97699999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":7.2249999999999996,"15min":7.2249999999999996},"last":1.0009999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80400000000000005,"5min":1.21,"15min":1.21},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82599999999999996,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.33400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.58099999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.32100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44}]},{"osd":3,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.74199999999999999},"min":{"1min":0.53000000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0840000000000001,"5min":1.298,"15min":1.298},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78800000000000003,"15min":0.753},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86899999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0800000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.52900000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97499999999999998,"5min":1.196,"15min":1.196},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":1.177},"last":0.81499999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.45700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.55800000000000005}]},{"osd":9,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.91300000000000003,"15min":0.88500000000000001},"min":{"1min":0.754,"5min":0.626,"15min":0.626},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.998,"15min":0.99099999999999999},"min":{"1min":0.83699999999999997,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.0920000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.57499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81299999999999994,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:26:49.628 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T06:26:50.128 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:50.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:50 smithi006 ceph-mon[31790]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-26T06:26:50.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:50 smithi006 ceph-mon[31790]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:26:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:50 smithi090 ceph-mon[35681]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-26T06:26:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:50 smithi090 ceph-mon[35681]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:26:50.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:50 smithi167 ceph-mon[36229]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-26T06:26:50.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:50 smithi167 ceph-mon[36229]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:26:52.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:52 smithi006 ceph-mon[31790]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-02-26T06:26:52.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 -- 172.21.15.6:0/684382318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 msgr2=0x7fcbd810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 --2- 172.21.15.6:0/684382318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd810eac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009fc0 tx=0x7fcbd000b330 comp rx=0 tx=0).stop 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 -- 172.21.15.6:0/684382318 shutdown_connections 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 --2- 172.21.15.6:0/684382318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd810eac0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 --2- 172.21.15.6:0/684382318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8107ca0 0x7fcbd8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 --2- 172.21.15.6:0/684382318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8101cc0 0x7fcbd8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 -- 172.21.15.6:0/684382318 >> 172.21.15.6:0/684382318 conn(0x7fcbd8078da0 msgr2=0x7fcbd80791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 -- 172.21.15.6:0/684382318 shutdown_connections 2024-02-26T06:26:52.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.310+0000 7fcbdda00700 1 -- 172.21.15.6:0/684382318 wait complete. 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.311+0000 7fcbdda00700 1 Processor -- start 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.311+0000 7fcbdda00700 1 -- start start 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbdda00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8101cc0 0x7fcbd8073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbdda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd80744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbdda00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 0x7fcbd80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbdda00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81a6570 con 0x7fcbd8107ca0 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbdda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81a66e0 con 0x7fcbd8102680 2024-02-26T06:26:52.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbdda00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81a68a0 con 0x7fcbd8101cc0 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbd7400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 0x7fcbd80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbd7400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 0x7fcbd80749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45562/0 (socket says 172.21.15.6:45562) 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbd7400700 1 -- 172.21.15.6:0/495554418 learned_addr learned my addr 172.21.15.6:0/495554418 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbd6a00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8101cc0 0x7fcbd8073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.312+0000 7fcbce000700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd80744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd6a00700 1 -- 172.21.15.6:0/495554418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 msgr2=0x7fcbd80744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd6a00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd80744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd6a00700 1 -- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 msgr2=0x7fcbd80749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd6a00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 0x7fcbd80749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd6a00700 1 -- 172.21.15.6:0/495554418 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00095f0 con 0x7fcbd8101cc0 2024-02-26T06:26:52.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd7400700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 0x7fcbd80749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbce000700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd80744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.313+0000 7fcbd6a00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8101cc0 0x7fcbd8073f70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800e9f0 tx=0x7fcbc800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.314+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc800c9c0 con 0x7fcbd8101cc0 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.314+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8010430 con 0x7fcbd8101cc0 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.314+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8018680 con 0x7fcbd8101cc0 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.314+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd0009c40 con 0x7fcbd8101cc0 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.314+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd81a6ea0 con 0x7fcbd8101cc0 2024-02-26T06:26:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.315+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd810a180 con 0x7fcbd8101cc0 2024-02-26T06:26:52.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.317+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fcbc8010910 con 0x7fcbd8101cc0 2024-02-26T06:26:52.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.318+0000 7fcbcfe00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcbbc0a21d0 0x7fcbbc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.318+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fcbc8014070 con 0x7fcbd8101cc0 2024-02-26T06:26:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.318+0000 7fcbce000700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcbbc0a21d0 0x7fcbbc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.319+0000 7fcbce000700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcbbc0a21d0 0x7fcbbc0a4690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8106a20 tx=0x7fcbc0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:52.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.322+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcbc8052d20 con 0x7fcbd8101cc0 2024-02-26T06:26:52.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:52 smithi090 ceph-mon[35681]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-02-26T06:26:52.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:52 smithi167 ceph-mon[36229]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-02-26T06:26:52.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.541+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbd8111d70 con 0x7fcbbc0a21d0 2024-02-26T06:26:52.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.572+0000 7fcbcfe00700 1 -- 172.21.15.6:0/495554418 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+342842 (secure 0 0 0) 0x7fcbd8111d70 con 0x7fcbbc0a21d0 2024-02-26T06:26:52.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:26:52.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcbbc0a21d0 msgr2=0x7fcbbc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:52.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcbbc0a21d0 0x7fcbbc0a4690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8106a20 tx=0x7fcbc0009310 comp rx=0 tx=0).stop 2024-02-26T06:26:52.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8101cc0 msgr2=0x7fcbd8073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:52.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8101cc0 0x7fcbd8073f70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800e9f0 tx=0x7fcbc800ed00 comp rx=0 tx=0).stop 2024-02-26T06:26:52.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 shutdown_connections 2024-02-26T06:26:52.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcbbc0a21d0 0x7fcbbc0a4690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8107ca0 0x7fcbd80749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8101cc0 0x7fcbd8073f70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 --2- 172.21.15.6:0/495554418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd8102680 0x7fcbd80744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 >> 172.21.15.6:0/495554418 conn(0x7fcbd8078da0 msgr2=0x7fcbd8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 shutdown_connections 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:52.581+0000 7fcbdda00700 1 -- 172.21.15.6:0/495554418 wait complete. 2024-02-26T06:26:52.581 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:26:53.623 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-02-26T06:26:52.145663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657828,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1029,"num_read_kb":639,"num_write":430,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181457,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383356,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558468,"statfs":{"total":1151940427776,"available":1151547871232,"internally_reserved":0,"allocated":13725696,"data_stored":7442388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004296"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125545+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:26:42.125545+0000","last_peered":"2024-02-26T06:26:42.125545+0000","last_clean":"2024-02-26T06:26:42.125545+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:26:42.125545+0000","last_undegraded":"2024-02-26T06:26:42.125545+0000","last_fullsized":"2024-02-26T06:26:42.125545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121536+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:26:42.121536+0000","last_peered":"2024-02-26T06:26:42.121536+0000","last_clean":"2024-02-26T06:26:42.121536+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:26:42.121536+0000","last_undegraded":"2024-02-26T06:26:42.121536+0000","last_fullsized":"2024-02-26T06:26:42.121536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128147+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:26:42.128147+0000","last_peered":"2024-02-26T06:26:42.128147+0000","last_clean":"2024-02-26T06:26:42.128147+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:26:42.128147+0000","last_undegraded":"2024-02-26T06:26:42.128147+0000","last_fullsized":"2024-02-26T06:26:42.128147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106826+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:26:42.106826+0000","last_peered":"2024-02-26T06:26:42.106826+0000","last_clean":"2024-02-26T06:26:42.106826+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:26:42.106826+0000","last_undegraded":"2024-02-26T06:26:42.106826+0000","last_fullsized":"2024-02-26T06:26:42.106826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101875+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:26:42.101875+0000","last_peered":"2024-02-26T06:26:42.101875+0000","last_clean":"2024-02-26T06:26:42.101875+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:26:42.101875+0000","last_undegraded":"2024-02-26T06:26:42.101875+0000","last_fullsized":"2024-02-26T06:26:42.101875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105400+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:26:42.105400+0000","last_peered":"2024-02-26T06:26:42.105400+0000","last_clean":"2024-02-26T06:26:42.105400+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:26:42.105400+0000","last_undegraded":"2024-02-26T06:26:42.105400+0000","last_fullsized":"2024-02-26T06:26:42.105400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114163+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:26:42.114163+0000","last_peered":"2024-02-26T06:26:42.114163+0000","last_clean":"2024-02-26T06:26:42.114163+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:26:42.114163+0000","last_undegraded":"2024-02-26T06:26:42.114163+0000","last_fullsized":"2024-02-26T06:26:42.114163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123188+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:26:42.123188+0000","last_peered":"2024-02-26T06:26:42.123188+0000","last_clean":"2024-02-26T06:26:42.123188+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:26:42.123188+0000","last_undegraded":"2024-02-26T06:26:42.123188+0000","last_fullsized":"2024-02-26T06:26:42.123188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113809+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:26:42.113809+0000","last_peered":"2024-02-26T06:26:42.113809+0000","last_clean":"2024-02-26T06:26:42.113809+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:26:42.113809+0000","last_undegraded":"2024-02-26T06:26:42.113809+0000","last_fullsized":"2024-02-26T06:26:42.113809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.104562+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:26:42.104562+0000","last_peered":"2024-02-26T06:26:42.104562+0000","last_clean":"2024-02-26T06:26:42.104562+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:26:42.104562+0000","last_undegraded":"2024-02-26T06:26:42.104562+0000","last_fullsized":"2024-02-26T06:26:42.104562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107477+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:26:42.107477+0000","last_peered":"2024-02-26T06:26:42.107477+0000","last_clean":"2024-02-26T06:26:42.107477+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:26:42.107477+0000","last_undegraded":"2024-02-26T06:26:42.107477+0000","last_fullsized":"2024-02-26T06:26:42.107477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100407+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:26:42.100407+0000","last_peered":"2024-02-26T06:26:42.100407+0000","last_clean":"2024-02-26T06:26:42.100407+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:26:42.100407+0000","last_undegraded":"2024-02-26T06:26:42.100407+0000","last_fullsized":"2024-02-26T06:26:42.100407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123361+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:26:42.123361+0000","last_peered":"2024-02-26T06:26:42.123361+0000","last_clean":"2024-02-26T06:26:42.123361+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:26:42.123361+0000","last_undegraded":"2024-02-26T06:26:42.123361+0000","last_fullsized":"2024-02-26T06:26:42.123361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127617+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:26:42.127617+0000","last_peered":"2024-02-26T06:26:42.127617+0000","last_clean":"2024-02-26T06:26:42.127617+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:26:42.127617+0000","last_undegraded":"2024-02-26T06:26:42.127617+0000","last_fullsized":"2024-02-26T06:26:42.127617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124207+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:26:42.124207+0000","last_peered":"2024-02-26T06:26:42.124207+0000","last_clean":"2024-02-26T06:26:42.124207+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:26:42.124207+0000","last_undegraded":"2024-02-26T06:26:42.124207+0000","last_fullsized":"2024-02-26T06:26:42.124207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129267+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:26:42.129267+0000","last_peered":"2024-02-26T06:26:42.129267+0000","last_clean":"2024-02-26T06:26:42.129267+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:26:42.129267+0000","last_undegraded":"2024-02-26T06:26:42.129267+0000","last_fullsized":"2024-02-26T06:26:42.129267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125702+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:26:42.125702+0000","last_peered":"2024-02-26T06:26:42.125702+0000","last_clean":"2024-02-26T06:26:42.125702+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:26:42.125702+0000","last_undegraded":"2024-02-26T06:26:42.125702+0000","last_fullsized":"2024-02-26T06:26:42.125702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107587+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:26:42.107587+0000","last_peered":"2024-02-26T06:26:42.107587+0000","last_clean":"2024-02-26T06:26:42.107587+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:26:42.107587+0000","last_undegraded":"2024-02-26T06:26:42.107587+0000","last_fullsized":"2024-02-26T06:26:42.107587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125040+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:26:42.125040+0000","last_peered":"2024-02-26T06:26:42.125040+0000","last_clean":"2024-02-26T06:26:42.125040+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:26:42.125040+0000","last_undegraded":"2024-02-26T06:26:42.125040+0000","last_fullsized":"2024-02-26T06:26:42.125040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:48.983883+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:26:48.983883+0000","last_peered":"2024-02-26T06:26:48.983883+0000","last_clean":"2024-02-26T06:26:48.983883+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:26:48.983883+0000","last_undegraded":"2024-02-26T06:26:48.983883+0000","last_fullsized":"2024-02-26T06:26:48.983883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120727+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:26:42.120727+0000","last_peered":"2024-02-26T06:26:42.120727+0000","last_clean":"2024-02-26T06:26:42.120727+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:26:42.120727+0000","last_undegraded":"2024-02-26T06:26:42.120727+0000","last_fullsized":"2024-02-26T06:26:42.120727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869401+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:26:13.869401+0000","last_peered":"2024-02-26T06:26:13.869401+0000","last_clean":"2024-02-26T06:26:13.869401+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:26:13.869401+0000","last_undegraded":"2024-02-26T06:26:13.869401+0000","last_fullsized":"2024-02-26T06:26:13.869401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128493+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:26:42.128493+0000","last_peered":"2024-02-26T06:26:42.128493+0000","last_clean":"2024-02-26T06:26:42.128493+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:26:42.128493+0000","last_undegraded":"2024-02-26T06:26:42.128493+0000","last_fullsized":"2024-02-26T06:26:42.128493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.864219+0000","last_change":"2024-02-26T06:26:13.864219+0000","last_active":"2024-02-26T06:26:13.864219+0000","last_peered":"2024-02-26T06:26:13.864219+0000","last_clean":"2024-02-26T06:26:13.864219+0000","last_became_active":"2024-02-26T06:26:13.851415+0000","last_became_peered":"2024-02-26T06:26:13.851415+0000","last_unstale":"2024-02-26T06:26:13.864219+0000","last_undegraded":"2024-02-26T06:26:13.864219+0000","last_fullsized":"2024-02-26T06:26:13.864219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:46:36.684794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125570+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:26:42.125570+0000","last_peered":"2024-02-26T06:26:42.125570+0000","last_clean":"2024-02-26T06:26:42.125570+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:26:42.125570+0000","last_undegraded":"2024-02-26T06:26:42.125570+0000","last_fullsized":"2024-02-26T06:26:42.125570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.874338+0000","last_change":"2024-02-26T06:26:13.873833+0000","last_active":"2024-02-26T06:26:13.874338+0000","last_peered":"2024-02-26T06:26:13.874338+0000","last_clean":"2024-02-26T06:26:13.874338+0000","last_became_active":"2024-02-26T06:26:13.852310+0000","last_became_peered":"2024-02-26T06:26:13.852310+0000","last_unstale":"2024-02-26T06:26:13.874338+0000","last_undegraded":"2024-02-26T06:26:13.874338+0000","last_fullsized":"2024-02-26T06:26:13.874338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:20.887692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.117029+0000","last_change":"2024-02-26T06:26:13.864818+0000","last_active":"2024-02-26T06:26:42.117029+0000","last_peered":"2024-02-26T06:26:42.117029+0000","last_clean":"2024-02-26T06:26:42.117029+0000","last_became_active":"2024-02-26T06:26:13.863821+0000","last_became_peered":"2024-02-26T06:26:13.863821+0000","last_unstale":"2024-02-26T06:26:42.117029+0000","last_undegraded":"2024-02-26T06:26:42.117029+0000","last_fullsized":"2024-02-26T06:26:42.117029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:38.266114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:47.916538+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:26:47.916538+0000","last_peered":"2024-02-26T06:26:47.916538+0000","last_clean":"2024-02-26T06:26:47.916538+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:26:47.916538+0000","last_undegraded":"2024-02-26T06:26:47.916538+0000","last_fullsized":"2024-02-26T06:26:47.916538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098741+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:26:42.098741+0000","last_peered":"2024-02-26T06:26:42.098741+0000","last_clean":"2024-02-26T06:26:42.098741+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:26:42.098741+0000","last_undegraded":"2024-02-26T06:26:42.098741+0000","last_fullsized":"2024-02-26T06:26:42.098741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851699+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:26:13.851699+0000","last_peered":"2024-02-26T06:26:13.851699+0000","last_clean":"2024-02-26T06:26:13.851699+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:26:13.851699+0000","last_undegraded":"2024-02-26T06:26:13.851699+0000","last_fullsized":"2024-02-26T06:26:13.851699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131059+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:26:42.131059+0000","last_peered":"2024-02-26T06:26:42.131059+0000","last_clean":"2024-02-26T06:26:42.131059+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:26:42.131059+0000","last_undegraded":"2024-02-26T06:26:42.131059+0000","last_fullsized":"2024-02-26T06:26:42.131059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.858204+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:26:13.858204+0000","last_peered":"2024-02-26T06:26:13.858204+0000","last_clean":"2024-02-26T06:26:13.858204+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:26:13.858204+0000","last_undegraded":"2024-02-26T06:26:13.858204+0000","last_fullsized":"2024-02-26T06:26:13.858204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128016+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:26:42.128016+0000","last_peered":"2024-02-26T06:26:42.128016+0000","last_clean":"2024-02-26T06:26:42.128016+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:26:42.128016+0000","last_undegraded":"2024-02-26T06:26:42.128016+0000","last_fullsized":"2024-02-26T06:26:42.128016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.850718+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:26:13.850718+0000","last_peered":"2024-02-26T06:26:13.850718+0000","last_clean":"2024-02-26T06:26:13.850718+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:26:13.850718+0000","last_undegraded":"2024-02-26T06:26:13.850718+0000","last_fullsized":"2024-02-26T06:26:13.850718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131552+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:26:42.131552+0000","last_peered":"2024-02-26T06:26:42.131552+0000","last_clean":"2024-02-26T06:26:42.131552+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:26:42.131552+0000","last_undegraded":"2024-02-26T06:26:42.131552+0000","last_fullsized":"2024-02-26T06:26:42.131552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":193,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:48.917287+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:26:48.917287+0000","last_peered":"2024-02-26T06:26:48.917287+0000","last_clean":"2024-02-26T06:26:48.917287+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:26:48.917287+0000","last_undegraded":"2024-02-26T06:26:48.917287+0000","last_fullsized":"2024-02-26T06:26:48.917287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.115681+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:26:42.115681+0000","last_peered":"2024-02-26T06:26:42.115681+0000","last_clean":"2024-02-26T06:26:42.115681+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:26:42.115681+0000","last_undegraded":"2024-02-26T06:26:42.115681+0000","last_fullsized":"2024-02-26T06:26:42.115681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848244+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:26:13.848244+0000","last_peered":"2024-02-26T06:26:13.848244+0000","last_clean":"2024-02-26T06:26:13.848244+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:26:13.848244+0000","last_undegraded":"2024-02-26T06:26:13.848244+0000","last_fullsized":"2024-02-26T06:26:13.848244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128168+0000","last_change":"2024-02-26T06:26:13.858594+0000","last_active":"2024-02-26T06:26:42.128168+0000","last_peered":"2024-02-26T06:26:42.128168+0000","last_clean":"2024-02-26T06:26:42.128168+0000","last_became_active":"2024-02-26T06:26:13.852999+0000","last_became_peered":"2024-02-26T06:26:13.852999+0000","last_unstale":"2024-02-26T06:26:42.128168+0000","last_undegraded":"2024-02-26T06:26:42.128168+0000","last_fullsized":"2024-02-26T06:26:42.128168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:17:51.940824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851762+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:26:13.851762+0000","last_peered":"2024-02-26T06:26:13.851762+0000","last_clean":"2024-02-26T06:26:13.851762+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:26:13.851762+0000","last_undegraded":"2024-02-26T06:26:13.851762+0000","last_fullsized":"2024-02-26T06:26:13.851762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125747+0000","last_change":"2024-02-26T06:26:13.863050+0000","last_active":"2024-02-26T06:26:42.125747+0000","last_peered":"2024-02-26T06:26:42.125747+0000","last_clean":"2024-02-26T06:26:42.125747+0000","last_became_active":"2024-02-26T06:26:13.862635+0000","last_became_peered":"2024-02-26T06:26:13.862635+0000","last_unstale":"2024-02-26T06:26:42.125747+0000","last_undegraded":"2024-02-26T06:26:42.125747+0000","last_fullsized":"2024-02-26T06:26:42.125747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:38:45.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":233,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:48.917242+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:26:48.917242+0000","last_peered":"2024-02-26T06:26:48.917242+0000","last_clean":"2024-02-26T06:26:48.917242+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:26:48.917242+0000","last_undegraded":"2024-02-26T06:26:48.917242+0000","last_fullsized":"2024-02-26T06:26:48.917242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":179,"num_read_kb":149,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099872+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:26:42.099872+0000","last_peered":"2024-02-26T06:26:42.099872+0000","last_clean":"2024-02-26T06:26:42.099872+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:26:42.099872+0000","last_undegraded":"2024-02-26T06:26:42.099872+0000","last_fullsized":"2024-02-26T06:26:42.099872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848072+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:26:13.848072+0000","last_peered":"2024-02-26T06:26:13.848072+0000","last_clean":"2024-02-26T06:26:13.848072+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:26:13.848072+0000","last_undegraded":"2024-02-26T06:26:13.848072+0000","last_fullsized":"2024-02-26T06:26:13.848072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131700+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:26:42.131700+0000","last_peered":"2024-02-26T06:26:42.131700+0000","last_clean":"2024-02-26T06:26:42.131700+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:26:42.131700+0000","last_undegraded":"2024-02-26T06:26:42.131700+0000","last_fullsized":"2024-02-26T06:26:42.131700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851099+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:26:13.851099+0000","last_peered":"2024-02-26T06:26:13.851099+0000","last_clean":"2024-02-26T06:26:13.851099+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:26:13.851099+0000","last_undegraded":"2024-02-26T06:26:13.851099+0000","last_fullsized":"2024-02-26T06:26:13.851099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112214+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:26:42.112214+0000","last_peered":"2024-02-26T06:26:42.112214+0000","last_clean":"2024-02-26T06:26:42.112214+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:26:42.112214+0000","last_undegraded":"2024-02-26T06:26:42.112214+0000","last_fullsized":"2024-02-26T06:26:42.112214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:47.985647+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:26:47.985647+0000","last_peered":"2024-02-26T06:26:47.985647+0000","last_clean":"2024-02-26T06:26:47.985647+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:26:47.985647+0000","last_undegraded":"2024-02-26T06:26:47.985647+0000","last_fullsized":"2024-02-26T06:26:47.985647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112955+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:26:42.112955+0000","last_peered":"2024-02-26T06:26:42.112955+0000","last_clean":"2024-02-26T06:26:42.112955+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:26:42.112955+0000","last_undegraded":"2024-02-26T06:26:42.112955+0000","last_fullsized":"2024-02-26T06:26:42.112955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844664+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:26:13.844664+0000","last_peered":"2024-02-26T06:26:13.844664+0000","last_clean":"2024-02-26T06:26:13.844664+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:26:13.844664+0000","last_undegraded":"2024-02-26T06:26:13.844664+0000","last_fullsized":"2024-02-26T06:26:13.844664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113707+0000","last_change":"2024-02-26T06:26:13.862455+0000","last_active":"2024-02-26T06:26:42.113707+0000","last_peered":"2024-02-26T06:26:42.113707+0000","last_clean":"2024-02-26T06:26:42.113707+0000","last_became_active":"2024-02-26T06:26:13.861385+0000","last_became_peered":"2024-02-26T06:26:13.861385+0000","last_unstale":"2024-02-26T06:26:42.113707+0000","last_undegraded":"2024-02-26T06:26:42.113707+0000","last_fullsized":"2024-02-26T06:26:42.113707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:42:41.345104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869092+0000","last_change":"2024-02-26T06:26:13.868540+0000","last_active":"2024-02-26T06:26:13.869092+0000","last_peered":"2024-02-26T06:26:13.869092+0000","last_clean":"2024-02-26T06:26:13.869092+0000","last_became_active":"2024-02-26T06:26:13.852149+0000","last_became_peered":"2024-02-26T06:26:13.852149+0000","last_unstale":"2024-02-26T06:26:13.869092+0000","last_undegraded":"2024-02-26T06:26:13.869092+0000","last_fullsized":"2024-02-26T06:26:13.869092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:14:30.745782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101245+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:26:42.101245+0000","last_peered":"2024-02-26T06:26:42.101245+0000","last_clean":"2024-02-26T06:26:42.101245+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:26:42.101245+0000","last_undegraded":"2024-02-26T06:26:42.101245+0000","last_fullsized":"2024-02-26T06:26:42.101245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.859894+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:26:13.859894+0000","last_peered":"2024-02-26T06:26:13.859894+0000","last_clean":"2024-02-26T06:26:13.859894+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:26:13.859894+0000","last_undegraded":"2024-02-26T06:26:13.859894+0000","last_fullsized":"2024-02-26T06:26:13.859894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.103414+0000","last_change":"2024-02-26T06:26:13.872244+0000","last_active":"2024-02-26T06:26:42.103414+0000","last_peered":"2024-02-26T06:26:42.103414+0000","last_clean":"2024-02-26T06:26:42.103414+0000","last_became_active":"2024-02-26T06:26:13.866094+0000","last_became_peered":"2024-02-26T06:26:13.866094+0000","last_unstale":"2024-02-26T06:26:42.103414+0000","last_undegraded":"2024-02-26T06:26:42.103414+0000","last_fullsized":"2024-02-26T06:26:42.103414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:56:52.301479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:48.984805+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:26:48.984805+0000","last_peered":"2024-02-26T06:26:48.984805+0000","last_clean":"2024-02-26T06:26:48.984805+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:26:48.984805+0000","last_undegraded":"2024-02-26T06:26:48.984805+0000","last_fullsized":"2024-02-26T06:26:48.984805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.102768+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:26:42.102768+0000","last_peered":"2024-02-26T06:26:42.102768+0000","last_clean":"2024-02-26T06:26:42.102768+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:26:42.102768+0000","last_undegraded":"2024-02-26T06:26:42.102768+0000","last_fullsized":"2024-02-26T06:26:42.102768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.093903+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:26:42.093903+0000","last_peered":"2024-02-26T06:26:42.093903+0000","last_clean":"2024-02-26T06:26:42.093903+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:26:42.093903+0000","last_undegraded":"2024-02-26T06:26:42.093903+0000","last_fullsized":"2024-02-26T06:26:42.093903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:48.987090+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:26:48.987090+0000","last_peered":"2024-02-26T06:26:48.987090+0000","last_clean":"2024-02-26T06:26:48.987090+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:26:48.987090+0000","last_undegraded":"2024-02-26T06:26:48.987090+0000","last_fullsized":"2024-02-26T06:26:48.987090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130301+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:26:42.130301+0000","last_peered":"2024-02-26T06:26:42.130301+0000","last_clean":"2024-02-26T06:26:42.130301+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:26:42.130301+0000","last_undegraded":"2024-02-26T06:26:42.130301+0000","last_fullsized":"2024-02-26T06:26:42.130301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849897+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:26:13.849897+0000","last_peered":"2024-02-26T06:26:13.849897+0000","last_clean":"2024-02-26T06:26:13.849897+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:26:13.849897+0000","last_undegraded":"2024-02-26T06:26:13.849897+0000","last_fullsized":"2024-02-26T06:26:13.849897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846775+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:26:13.846775+0000","last_peered":"2024-02-26T06:26:13.846775+0000","last_clean":"2024-02-26T06:26:13.846775+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:26:13.846775+0000","last_undegraded":"2024-02-26T06:26:13.846775+0000","last_fullsized":"2024-02-26T06:26:13.846775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120068+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:26:42.120068+0000","last_peered":"2024-02-26T06:26:42.120068+0000","last_clean":"2024-02-26T06:26:42.120068+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:26:42.120068+0000","last_undegraded":"2024-02-26T06:26:42.120068+0000","last_fullsized":"2024-02-26T06:26:42.120068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846836+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:26:13.846836+0000","last_peered":"2024-02-26T06:26:13.846836+0000","last_clean":"2024-02-26T06:26:13.846836+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:26:13.846836+0000","last_undegraded":"2024-02-26T06:26:13.846836+0000","last_fullsized":"2024-02-26T06:26:13.846836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.845030+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:26:13.845030+0000","last_peered":"2024-02-26T06:26:13.845030+0000","last_clean":"2024-02-26T06:26:13.845030+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:26:13.845030+0000","last_undegraded":"2024-02-26T06:26:13.845030+0000","last_fullsized":"2024-02-26T06:26:13.845030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.122929+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:26:42.122929+0000","last_peered":"2024-02-26T06:26:42.122929+0000","last_clean":"2024-02-26T06:26:42.122929+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:26:42.122929+0000","last_undegraded":"2024-02-26T06:26:42.122929+0000","last_fullsized":"2024-02-26T06:26:42.122929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'14","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.860053+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:26:13.860053+0000","last_peered":"2024-02-26T06:26:13.860053+0000","last_clean":"2024-02-26T06:26:13.860053+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:26:13.860053+0000","last_undegraded":"2024-02-26T06:26:13.860053+0000","last_fullsized":"2024-02-26T06:26:13.860053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120512+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:26:42.120512+0000","last_peered":"2024-02-26T06:26:42.120512+0000","last_clean":"2024-02-26T06:26:42.120512+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:26:42.120512+0000","last_undegraded":"2024-02-26T06:26:42.120512+0000","last_fullsized":"2024-02-26T06:26:42.120512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849919+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:26:13.849919+0000","last_peered":"2024-02-26T06:26:13.849919+0000","last_clean":"2024-02-26T06:26:13.849919+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:26:13.849919+0000","last_undegraded":"2024-02-26T06:26:13.849919+0000","last_fullsized":"2024-02-26T06:26:13.849919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128451+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:26:42.128451+0000","last_peered":"2024-02-26T06:26:42.128451+0000","last_clean":"2024-02-26T06:26:42.128451+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:26:42.128451+0000","last_undegraded":"2024-02-26T06:26:42.128451+0000","last_fullsized":"2024-02-26T06:26:42.128451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.847060+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:26:13.847060+0000","last_peered":"2024-02-26T06:26:13.847060+0000","last_clean":"2024-02-26T06:26:13.847060+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:26:13.847060+0000","last_undegraded":"2024-02-26T06:26:13.847060+0000","last_fullsized":"2024-02-26T06:26:13.847060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.069585+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:26:42.069585+0000","last_peered":"2024-02-26T06:26:42.069585+0000","last_clean":"2024-02-26T06:26:42.069585+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:26:42.069585+0000","last_undegraded":"2024-02-26T06:26:42.069585+0000","last_fullsized":"2024-02-26T06:26:42.069585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.866190+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:26:13.866190+0000","last_peered":"2024-02-26T06:26:13.866190+0000","last_clean":"2024-02-26T06:26:13.866190+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:26:13.866190+0000","last_undegraded":"2024-02-26T06:26:13.866190+0000","last_fullsized":"2024-02-26T06:26:13.866190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105913+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:26:42.105913+0000","last_peered":"2024-02-26T06:26:42.105913+0000","last_clean":"2024-02-26T06:26:42.105913+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:26:42.105913+0000","last_undegraded":"2024-02-26T06:26:42.105913+0000","last_fullsized":"2024-02-26T06:26:42.105913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848263+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:26:13.848263+0000","last_peered":"2024-02-26T06:26:13.848263+0000","last_clean":"2024-02-26T06:26:13.848263+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:26:13.848263+0000","last_undegraded":"2024-02-26T06:26:13.848263+0000","last_fullsized":"2024-02-26T06:26:13.848263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129595+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:26:42.129595+0000","last_peered":"2024-02-26T06:26:42.129595+0000","last_clean":"2024-02-26T06:26:42.129595+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:26:42.129595+0000","last_undegraded":"2024-02-26T06:26:42.129595+0000","last_fullsized":"2024-02-26T06:26:42.129595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844759+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:26:13.844759+0000","last_peered":"2024-02-26T06:26:13.844759+0000","last_clean":"2024-02-26T06:26:13.844759+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:26:13.844759+0000","last_undegraded":"2024-02-26T06:26:13.844759+0000","last_fullsized":"2024-02-26T06:26:13.844759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.111151+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:26:42.111151+0000","last_peered":"2024-02-26T06:26:42.111151+0000","last_clean":"2024-02-26T06:26:42.111151+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:26:42.111151+0000","last_undegraded":"2024-02-26T06:26:42.111151+0000","last_fullsized":"2024-02-26T06:26:42.111151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:49.919527+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:26:49.919527+0000","last_peered":"2024-02-26T06:26:49.919527+0000","last_clean":"2024-02-26T06:26:49.919527+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:26:49.919527+0000","last_undegraded":"2024-02-26T06:26:49.919527+0000","last_fullsized":"2024-02-26T06:26:49.919527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107584+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:26:42.107584+0000","last_peered":"2024-02-26T06:26:42.107584+0000","last_clean":"2024-02-26T06:26:42.107584+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:26:42.107584+0000","last_undegraded":"2024-02-26T06:26:42.107584+0000","last_fullsized":"2024-02-26T06:26:42.107584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851618+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:26:13.851618+0000","last_peered":"2024-02-26T06:26:13.851618+0000","last_clean":"2024-02-26T06:26:13.851618+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:26:13.851618+0000","last_undegraded":"2024-02-26T06:26:13.851618+0000","last_fullsized":"2024-02-26T06:26:13.851618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106886+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:26:42.106886+0000","last_peered":"2024-02-26T06:26:42.106886+0000","last_clean":"2024-02-26T06:26:42.106886+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:26:42.106886+0000","last_undegraded":"2024-02-26T06:26:42.106886+0000","last_fullsized":"2024-02-26T06:26:42.106886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.878533+0000","last_change":"2024-02-26T06:26:13.878533+0000","last_active":"2024-02-26T06:26:13.878533+0000","last_peered":"2024-02-26T06:26:13.878533+0000","last_clean":"2024-02-26T06:26:13.878533+0000","last_became_active":"2024-02-26T06:26:13.866478+0000","last_became_peered":"2024-02-26T06:26:13.866478+0000","last_unstale":"2024-02-26T06:26:13.878533+0000","last_undegraded":"2024-02-26T06:26:13.878533+0000","last_fullsized":"2024-02-26T06:26:13.878533+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:00:23.896436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099199+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:26:42.099199+0000","last_peered":"2024-02-26T06:26:42.099199+0000","last_clean":"2024-02-26T06:26:42.099199+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:26:42.099199+0000","last_undegraded":"2024-02-26T06:26:42.099199+0000","last_fullsized":"2024-02-26T06:26:42.099199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099772+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:26:42.099772+0000","last_peered":"2024-02-26T06:26:42.099772+0000","last_clean":"2024-02-26T06:26:42.099772+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:26:42.099772+0000","last_undegraded":"2024-02-26T06:26:42.099772+0000","last_fullsized":"2024-02-26T06:26:42.099772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.108107+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:26:42.108107+0000","last_peered":"2024-02-26T06:26:42.108107+0000","last_clean":"2024-02-26T06:26:42.108107+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:26:42.108107+0000","last_undegraded":"2024-02-26T06:26:42.108107+0000","last_fullsized":"2024-02-26T06:26:42.108107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123861+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:26:42.123861+0000","last_peered":"2024-02-26T06:26:42.123861+0000","last_clean":"2024-02-26T06:26:42.123861+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:26:42.123861+0000","last_undegraded":"2024-02-26T06:26:42.123861+0000","last_fullsized":"2024-02-26T06:26:42.123861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127224+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:26:42.127224+0000","last_peered":"2024-02-26T06:26:42.127224+0000","last_clean":"2024-02-26T06:26:42.127224+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:26:42.127224+0000","last_undegraded":"2024-02-26T06:26:42.127224+0000","last_fullsized":"2024-02-26T06:26:42.127224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126738+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:26:42.126738+0000","last_peered":"2024-02-26T06:26:42.126738+0000","last_clean":"2024-02-26T06:26:42.126738+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:26:42.126738+0000","last_undegraded":"2024-02-26T06:26:42.126738+0000","last_fullsized":"2024-02-26T06:26:42.126738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.096657+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:26:42.096657+0000","last_peered":"2024-02-26T06:26:42.096657+0000","last_clean":"2024-02-26T06:26:42.096657+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:26:42.096657+0000","last_undegraded":"2024-02-26T06:26:42.096657+0000","last_fullsized":"2024-02-26T06:26:42.096657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106173+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:26:42.106173+0000","last_peered":"2024-02-26T06:26:42.106173+0000","last_clean":"2024-02-26T06:26:42.106173+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:26:42.106173+0000","last_undegraded":"2024-02-26T06:26:42.106173+0000","last_fullsized":"2024-02-26T06:26:42.106173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112171+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:26:42.112171+0000","last_peered":"2024-02-26T06:26:42.112171+0000","last_clean":"2024-02-26T06:26:42.112171+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:26:42.112171+0000","last_undegraded":"2024-02-26T06:26:42.112171+0000","last_fullsized":"2024-02-26T06:26:42.112171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130606+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:26:42.130606+0000","last_peered":"2024-02-26T06:26:42.130606+0000","last_clean":"2024-02-26T06:26:42.130606+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:26:42.130606+0000","last_undegraded":"2024-02-26T06:26:42.130606+0000","last_fullsized":"2024-02-26T06:26:42.130606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120820+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:26:42.120820+0000","last_peered":"2024-02-26T06:26:42.120820+0000","last_clean":"2024-02-26T06:26:42.120820+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:26:42.120820+0000","last_undegraded":"2024-02-26T06:26:42.120820+0000","last_fullsized":"2024-02-26T06:26:42.120820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131628+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:26:42.131628+0000","last_peered":"2024-02-26T06:26:42.131628+0000","last_clean":"2024-02-26T06:26:42.131628+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:26:42.131628+0000","last_undegraded":"2024-02-26T06:26:42.131628+0000","last_fullsized":"2024-02-26T06:26:42.131628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129177+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:26:42.129177+0000","last_peered":"2024-02-26T06:26:42.129177+0000","last_clean":"2024-02-26T06:26:42.129177+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:26:42.129177+0000","last_undegraded":"2024-02-26T06:26:42.129177+0000","last_fullsized":"2024-02-26T06:26:42.129177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100619+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:26:42.100619+0000","last_peered":"2024-02-26T06:26:42.100619+0000","last_clean":"2024-02-26T06:26:42.100619+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:26:42.100619+0000","last_undegraded":"2024-02-26T06:26:42.100619+0000","last_fullsized":"2024-02-26T06:26:42.100619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67226,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":893,"num_read_kb":520,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383715,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.62}]},{"osd":2,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.33500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63500000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.5},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72399999999999998,"5min":0.751,"15min":0.751},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.874,"15min":0.874},"last":0.43099999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0680000000000001,"5min":1.103,"15min":1.103},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.91000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3779999999999999,"5min":1.385,"15min":1.385},"last":0.64700000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.58999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.90100000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.65300000000000002},"min":{"1min":0.53000000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.80300000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54400000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65900000000000003}]},{"osd":10,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71099999999999997,"15min":0.69999999999999996},"min":{"1min":0.61099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.044,"15min":1.044},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.63600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.97799999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.84699999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1108767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.7549999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.78900000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.627}]},{"osd":2,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82099999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88700000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.117,"5min":1.458,"15min":1.458},"last":0.88600000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.70599999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.72199999999999998,"15min":0.73899999999999999},"min":{"1min":0.497,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80300000000000005,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.84999999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.85399999999999998,"15min":0.86599999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.069,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.081},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.145,"15min":1.145},"last":0.98299999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73799999999999999,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.82499999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84999999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.82399999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1830000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.97299999999999998,"15min":0.99099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.256},"last":0.85199999999999998}]},{"osd":11,"last update":"Mon Feb 26 06:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93000000000000005,"5min":1.014,"15min":1.014},"last":0.86299999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.4079999999999999,"15min":1.4330000000000001},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":2.1709999999999998,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.444,"15min":1.454},"min":{"1min":0.72099999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9239999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.7889999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1419999999999999,"15min":1.1890000000000001},"min":{"1min":0.75900000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.621,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.306},"min":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7450000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1279999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.5089999999999999,"15min":1.5569999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.1819999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.359,"15min":1.401},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.919,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.1879999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.681,"15min":1.7569999999999999},"min":{"1min":1.0169999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.601,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.46},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.8100000000000001,"15min":1.8620000000000001},"min":{"1min":0.79400000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.242,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":1.7050000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.694,"15min":1.772},"min":{"1min":0.94199999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.639,"5min":1.8540000000000001,"15min":1.9450000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.4409999999999998,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.238}]},{"osd":5,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.736,"5min":2.0779999999999998,"15min":2.0920000000000001},"min":{"1min":0.92300000000000004,"5min":0.879,"15min":0.879},"max":{"1min":2.4460000000000002,"5min":14.581,"15min":14.581},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.774,"5min":2.5979999999999999,"15min":2.8300000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.867,"5min":18.216000000000001,"15min":18.216000000000001},"last":1.6359999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0550000000000002,"5min":2.202,"15min":2.2879999999999998},"min":{"1min":1.175,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.762,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":2.335},{"interface":"front","average":{"1min":1.9850000000000001,"5min":2.1269999999999998,"15min":2.2250000000000001},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.98,"5min":3.044,"15min":3.044},"last":1.8240000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8420000000000001,"5min":2.012,"15min":2.0880000000000001},"min":{"1min":0.95799999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.4289999999999998,"5min":3.036,"15min":3.036},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.972,"5min":2.121,"15min":2.1899999999999999},"min":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.5630000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":2.2010000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":2.2210000000000001,"15min":2.3159999999999998},"min":{"1min":1.147,"5min":1.0469999999999999,"15min":1.0469999999999999},"max":{"1min":2.6659999999999999,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.6160000000000001},{"interface":"front","average":{"1min":1.9970000000000001,"5min":2.0920000000000001,"15min":2.1800000000000002},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":3.097,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":1.7470000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":1.895,"5min":1.954,"15min":1.964},"min":{"1min":1.026,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.544,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":1.964},{"interface":"front","average":{"1min":2.1160000000000001,"5min":2.1899999999999999,"15min":2.202},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":2.4420000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.71099999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.73399999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.83299999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.439},"min":{"1min":0.25800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.439,"15min":0.436},"min":{"1min":0.311,"5min":0.224,"15min":0.224},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0720000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.157},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.876,"15min":0.876},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71399999999999997,"5min":0.876,"15min":0.876},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73499999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.008,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.90400000000000003,"15min":0.92300000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0309999999999999,"5min":1.373,"15min":1.373},"last":0.98499999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":1.048,"15min":1.048},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:26:36 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1859999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.97399999999999998,"15min":1.0069999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.165,"5min":1.429,"15min":1.429},"last":1.1399999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88500000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.754}]},{"osd":11,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73499999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1049901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42299999999999999,"15min":0.40400000000000003},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.42899999999999999},"min":{"1min":0.36399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.60399999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56000000000000005}]},{"osd":3,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.98599999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0029999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.79400000000000004}]},{"osd":4,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.5,"15min":0.495},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.625,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61599999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.44800000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80500000000000005,"5min":0.877,"15min":0.877},"last":0.60999999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:26:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83999999999999997,"15min":0.84399999999999997},"min":{"1min":0.60699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93899999999999995,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":0.91700000000000004,"15min":0.872},"min":{"1min":0.62,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":6.9900000000000002,"5min":6.9900000000000002,"15min":6.9900000000000002},"last":0.73599999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.746},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.98199999999999998,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0680000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.089,"15min":1.046},"min":{"1min":0.61499999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.0380000000000003,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.90200000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84299999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70899999999999996}]},{"osd":11,"last update":"Mon Feb 26 06:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77200000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.206,"15min":1.198},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.298,"15min":1.304},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.2080000000000002,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.431}]},{"osd":1,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0589999999999999,"15min":1.032},"min":{"1min":0.70799999999999996,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.097,"5min":1.1299999999999999,"15min":1.1100000000000001},"min":{"1min":0.78800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.9279999999999999,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.161}]},{"osd":2,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.143},"min":{"1min":0.59399999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6930000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.276,"15min":1.228},"min":{"1min":0.65400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.0209999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.274}]},{"osd":3,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5469999999999999,"15min":1.5269999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.4780000000000002,"5min":3.113,"15min":3.113},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.466,"5min":1.629,"15min":1.649},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.258,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.2330000000000001}]},{"osd":4,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3720000000000001,"15min":1.3560000000000001},"min":{"1min":0.65200000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0009999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.552,"15min":1.506},"min":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2410000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.5509999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:26:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.575,"15min":1.617},"min":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.4239999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.607,"15min":1.6000000000000001},"min":{"1min":0.67400000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.3319999999999999,"5min":2.798,"15min":2.798},"last":0.96299999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.6759999999999999,"15min":1.5489999999999999},"min":{"1min":0.85799999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.488,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":1.756},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.6659999999999999,"15min":1.6120000000000001},"min":{"1min":0.78700000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.5459999999999998,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":1.411}]},{"osd":8,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":1.782,"15min":1.726},"min":{"1min":0.86299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5720000000000001,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.76,"15min":1.742},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6970000000000001,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":1.472}]},{"osd":9,"last update":"Mon Feb 26 06:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":2.1059999999999999,"15min":2.2559999999999998},"min":{"1min":0.98299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.395,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.8839999999999999,"5min":2.2160000000000002,"15min":2.3580000000000001},"min":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.6349999999999998,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.8620000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:25:54 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":2.0680000000000001,"15min":2.1280000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.5779999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.988,"15min":2.0150000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.5640000000000001,"5min":2.806,"15min":2.806},"last":1.373}]}]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":517766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.627},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.374},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0740000000000001,"5min":1.252,"15min":1.252},"last":0.88300000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Mon Feb 26 06:26:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81599999999999995,"15min":0.80100000000000005},"min":{"1min":0.55200000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1639999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.04}]},{"osd":4,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.34599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.43099999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63600000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0669999999999999,"5min":1.196,"15min":1.196},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81200000000000006}]},{"osd":7,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70299999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61899999999999999}]},{"osd":8,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.52900000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.57599999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90100000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.752}]},{"osd":9,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78200000000000003}]},{"osd":10,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70599999999999996}]},{"osd":11,"last update":"Mon Feb 26 06:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.83999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":457920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92100000000000004,"15min":0.79800000000000004},"min":{"1min":0.56599999999999995,"5min":0.51500000000000001,"15min":0.45900000000000002},"max":{"1min":1.2869999999999999,"5min":1.569,"15min":1.569},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.81699999999999995},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.40899999999999997},"max":{"1min":1.3959999999999999,"5min":3.516,"15min":3.516},"last":1.196}]},{"osd":2,"last update":"Mon Feb 26 06:26:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.01,"15min":0.90000000000000002},"min":{"1min":0.56499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5049999999999999,"5min":1.667,"15min":1.667},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.1619999999999999,"15min":0.97999999999999998},"min":{"1min":0.495,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.724,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.278},{"interface":"front","average":{"1min":1.798,"5min":1.798,"15min":1.798},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":1.0289999999999999}]},{"osd":4,"last update":"Mon Feb 26 06:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.413,"15min":1.3340000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.411,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.236},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.52900000000000003,"5min":0.499,"15min":0.499},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.054}]},{"osd":5,"last update":"Mon Feb 26 06:26:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.345,"15min":1.3149999999999999},"min":{"1min":0.71799999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1459999999999999,"5min":2.653,"15min":2.653},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.6619999999999999,"15min":1.54},"min":{"1min":1.028,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.4610000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.488,"15min":1.4379999999999999},"min":{"1min":0.65400000000000003,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.4889999999999999,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.631,"5min":1.673,"15min":1.569},"min":{"1min":0.84699999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.6619999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.4750000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:25:57 2024","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.698,"15min":1.667},"min":{"1min":0.90900000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3090000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.8660000000000001,"5min":1.843,"15min":1.7989999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.5589999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":2.1080000000000001,"15min":2.1989999999999998},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.774,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.907},{"interface":"front","average":{"1min":2.1059999999999999,"5min":2.2090000000000001,"15min":2.2850000000000001},"min":{"1min":1.071,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.621}]},{"osd":10,"last update":"Mon Feb 26 06:25:57 2024","interfaces":[{"interface":"back","average":{"1min":1.9810000000000001,"5min":2.1219999999999999,"15min":2.1459999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.593},{"interface":"front","average":{"1min":1.494,"5min":1.702,"15min":1.7370000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.3140000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.4399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":1.861,"5min":1.8720000000000001,"15min":1.8740000000000001},"min":{"1min":0.97999999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.681,"15min":1.6850000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3690000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3999999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.40600000000000003},"max":{"1min":0.95999999999999996,"5min":1.163,"15min":1.163},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71899999999999997,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0900000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.74399999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.38600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.73799999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.35499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69099999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.78400000000000003,"15min":0.71599999999999997},"min":{"1min":0.497,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.087,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73399999999999999,"15min":0.69199999999999995},"min":{"1min":0.57699999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.169}]},{"osd":4,"last update":"Mon Feb 26 06:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.38600000000000001},"min":{"1min":0.36199999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.44500000000000001,"15min":0.42299999999999999},"min":{"1min":0.39300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51900000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.67800000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88,"15min":0.85699999999999998},"min":{"1min":0.69599999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0529999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.624},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.66200000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.63200000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.94599999999999995,"15min":0.96199999999999997},"min":{"1min":0.66900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.194,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.54400000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81799999999999995,"5min":0.875,"15min":0.875},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84399999999999997,"5min":1.089,"15min":1.089},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Feb 26 06:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.80800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":456289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.165,"15min":0.96299999999999997},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.867,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.0249999999999999},"min":{"1min":0.45400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0499999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.3420000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0980000000000001,"15min":0.91400000000000003},"min":{"1min":0.78500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.417,"5min":1.157,"15min":0.97999999999999998},"min":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":6.8369999999999997,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2230000000000001,"15min":1.0609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2789999999999999,"15min":1.125},"min":{"1min":0.67800000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2849999999999999,"15min":1.1060000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.0019999999999998,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.421,"15min":1.2470000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.0129999999999999,"5min":3.323,"15min":3.323},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.498,"15min":1.3580000000000001},"min":{"1min":0.91600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.8479999999999999,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":1.272},{"interface":"front","average":{"1min":1.548,"5min":1.5549999999999999,"15min":1.5089999999999999},"min":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.1349999999999998,"5min":5.7770000000000001,"15min":5.7770000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.72,"5min":1.8300000000000001,"15min":1.74},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":3.3159999999999998,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.504},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.643,"15min":1.49},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":3.1760000000000002,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.762,"15min":1.706},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.544,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.4750000000000001},{"interface":"front","average":{"1min":2.0129999999999999,"5min":1.911,"15min":1.8520000000000001},"min":{"1min":0.82699999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.758,"5min":6.758,"15min":6.758},"last":0.75}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:26:29 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":2.073,"15min":2.1739999999999999},"min":{"1min":0.73499999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":2.0030000000000001,"5min":2.1749999999999998,"15min":2.266},"min":{"1min":1.0820000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.9729999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":1.4099999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":2.1419999999999999,"15min":2.202},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.5049999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.6719999999999999,"5min":2.1160000000000001,"15min":2.1899999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.2349999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.76600000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:26:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.573},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0510000000000002,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":0.79600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.312,"5min":0.219,"15min":0.219},"max":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.29499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.35499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":1.04,"15min":1.04},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.71199999999999997},"min":{"1min":0.63200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.024,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.76800000000000002,"15min":0.70799999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0620000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60499999999999998,"15min":0.56100000000000005},"min":{"1min":0.45000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81899999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.55900000000000005},"min":{"1min":0.48099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84799999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.82299999999999995,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.036,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98999999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.501}]},{"osd":7,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.49199999999999999},"min":{"1min":0.40100000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.875,"15min":0.875},"last":0.60199999999999998}]},{"osd":8,"last update":"Mon Feb 26 06:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.65500000000000003},"min":{"1min":0.48899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88,"15min":0.90000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.997,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.054,"5min":1.665,"15min":1.665},"last":0.69599999999999995}]},{"osd":10,"last update":"Mon Feb 26 06:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67300000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81899999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47099999999999997}]},{"osd":11,"last update":"Mon Feb 26 06:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.624},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1049047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0489999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82599999999999996,"15min":0.97699999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":7.2249999999999996,"15min":7.2249999999999996},"last":0.752}]},{"osd":1,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80400000000000005,"5min":1.21,"15min":1.21},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82599999999999996,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.5}]},{"osd":2,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.33400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.58099999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.32100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.377}]},{"osd":3,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.74199999999999999},"min":{"1min":0.53000000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0840000000000001,"5min":1.298,"15min":1.298},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78800000000000003,"15min":0.753},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86899999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0800000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.52900000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97499999999999998,"5min":1.196,"15min":1.196},"last":0.623},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":1.177},"last":0.66300000000000003}]},{"osd":8,"last update":"Mon Feb 26 06:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.45700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47999999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:26:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.91300000000000003,"15min":0.88500000000000001},"min":{"1min":0.754,"5min":0.626,"15min":0.626},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.998,"15min":0.99099999999999999},"min":{"1min":0.83699999999999997,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.997}]},{"osd":10,"last update":"Mon Feb 26 06:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.57499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81299999999999994,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:26:53.630 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-26T06:26:53.630 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-26T06:26:53.630 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-26T06:26:53.630 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph health --format=json 2024-02-26T06:26:54.140 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:26:54.295 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:54 smithi006 ceph-mon[31790]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-02-26T06:26:54.296 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:54 smithi006 ceph-mon[31790]: from='client.24907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:26:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:54 smithi090 ceph-mon[35681]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-02-26T06:26:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:54 smithi090 ceph-mon[35681]: from='client.24907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:26:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:54 smithi167 ceph-mon[36229]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-02-26T06:26:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:54 smithi167 ceph-mon[36229]: from='client.24907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:26:56.104 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:56 smithi006 ceph-mon[31790]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 -- 172.21.15.6:0/3135469441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a681040c0 msgr2=0x7f3a681044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 --2- 172.21.15.6:0/3135469441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a681040c0 0x7f3a681044a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009990 tx=0x7f3a58009ca0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 -- 172.21.15.6:0/3135469441 shutdown_connections 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 --2- 172.21.15.6:0/3135469441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a681091a0 0x7f3a68110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 --2- 172.21.15.6:0/3135469441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a681040c0 0x7f3a681044a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 --2- 172.21.15.6:0/3135469441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a68104a70 0x7f3a68108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 -- 172.21.15.6:0/3135469441 >> 172.21.15.6:0/3135469441 conn(0x7f3a680ff950 msgr2=0x7f3a68101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 -- 172.21.15.6:0/3135469441 shutdown_connections 2024-02-26T06:26:56.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.274+0000 7f3a6d400700 1 -- 172.21.15.6:0/3135469441 wait complete. 2024-02-26T06:26:56.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.275+0000 7f3a6d400700 1 Processor -- start 2024-02-26T06:26:56.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 -- start start 2024-02-26T06:26:56.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a681040c0 0x7f3a6819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:56.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 0x7f3a6819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:56.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a681091a0 0x7f3a6819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:56.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819f310 con 0x7f3a681091a0 2024-02-26T06:26:56.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819f480 con 0x7f3a68104a70 2024-02-26T06:26:56.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a6d400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819f640 con 0x7f3a681040c0 2024-02-26T06:26:56.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 0x7f3a6819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 0x7f3a6819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:35874/0 (socket says 172.21.15.6:35874) 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a65e00700 1 -- 172.21.15.6:0/235794940 learned_addr learned my addr 172.21.15.6:0/235794940 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a66800700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a681040c0 0x7f3a6819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.276+0000 7f3a67200700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a681091a0 0x7f3a6819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a65e00700 1 -- 172.21.15.6:0/235794940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a681040c0 msgr2=0x7f3a6819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a65e00700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a681040c0 0x7f3a6819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a65e00700 1 -- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a681091a0 msgr2=0x7f3a6819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a65e00700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a681091a0 0x7f3a6819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a65e00700 1 -- 172.21.15.6:0/235794940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580095f0 con 0x7f3a68104a70 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a65e00700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 0x7f3a6819a970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400e970 tx=0x7f3a5400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.277+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54009790 con 0x7f3a68104a70 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.278+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a5400c7e0 con 0x7f3a68104a70 2024-02-26T06:26:56.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.278+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54005580 con 0x7f3a68104a70 2024-02-26T06:26:56.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.278+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6819f920 con 0x7f3a68104a70 2024-02-26T06:26:56.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.278+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a681a7180 con 0x7f3a68104a70 2024-02-26T06:26:56.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.279+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6810c3d0 con 0x7f3a68104a70 2024-02-26T06:26:56.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.281+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f3a5400cce0 con 0x7f3a68104a70 2024-02-26T06:26:56.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.282+0000 7f3a5f400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3a4c0a21d0 0x7f3a4c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:56.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.282+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3a54014070 con 0x7f3a68104a70 2024-02-26T06:26:56.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.282+0000 7f3a66800700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3a4c0a21d0 0x7f3a4c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:56.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.283+0000 7f3a66800700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3a4c0a21d0 0x7f3a4c0a4690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800b2e0 tx=0x7f3a58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:56.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.287+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a54096dd0 con 0x7f3a68104a70 2024-02-26T06:26:56.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:56 smithi167 ceph-mon[36229]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-26T06:26:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:56 smithi090 ceph-mon[35681]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-26T06:26:56.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.558+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3a6819b7f0 con 0x7f3a68104a70 2024-02-26T06:26:56.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.558+0000 7f3a5f400700 1 -- 172.21.15.6:0/235794940 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3a5409a490 con 0x7f3a68104a70 2024-02-26T06:26:56.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:26:56.558 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-26T06:26:56.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3a4c0a21d0 msgr2=0x7f3a4c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:56.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3a4c0a21d0 0x7f3a4c0a4690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800b2e0 tx=0x7f3a58005fd0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 msgr2=0x7f3a6819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 0x7f3a6819a970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400e970 tx=0x7f3a5400ec80 comp rx=0 tx=0).stop 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 shutdown_connections 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f3a4c0a21d0 0x7f3a4c0a4690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a681091a0 0x7f3a6819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a681040c0 0x7f3a6819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 --2- 172.21.15.6:0/235794940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a68104a70 0x7f3a6819a970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.561+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 >> 172.21.15.6:0/235794940 conn(0x7f3a680ff950 msgr2=0x7f3a68101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.562+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 shutdown_connections 2024-02-26T06:26:56.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:56.562+0000 7f3a6d400700 1 -- 172.21.15.6:0/235794940 wait complete. 2024-02-26T06:26:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:57 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/235794940' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T06:26:57.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:57 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/235794940' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T06:26:57.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:57 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/235794940' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T06:26:57.505 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-26T06:26:57.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 -- 172.21.15.6:0/1591437963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102260 msgr2=0x7ff8041026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 --2- 172.21.15.6:0/1591437963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102260 0x7ff8041026e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009900 tx=0x7ff7f402bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 -- 172.21.15.6:0/1591437963 shutdown_connections 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 --2- 172.21.15.6:0/1591437963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804102c20 0x7ff8041094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 --2- 172.21.15.6:0/1591437963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102260 0x7ff8041026e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 --2- 172.21.15.6:0/1591437963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff804101070 0x7ff804101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 -- 172.21.15.6:0/1591437963 >> 172.21.15.6:0/1591437963 conn(0x7ff8040fc5e0 msgr2=0x7ff8040fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:57.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 -- 172.21.15.6:0/1591437963 shutdown_connections 2024-02-26T06:26:57.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.742+0000 7ff80b200640 1 -- 172.21.15.6:0/1591437963 wait complete. 2024-02-26T06:26:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.743+0000 7ff80b200640 1 Processor -- start 2024-02-26T06:26:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.743+0000 7ff80b200640 1 -- start start 2024-02-26T06:26:57.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804101070 0x7ff804074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff804102260 0x7ff804073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80b200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 0x7ff804073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80b200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804074210 con 0x7ff804102c20 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804074380 con 0x7ff804102260 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8041a1170 con 0x7ff804101070 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 0x7ff804073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 0x7ff804073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38214/0 (socket says 172.21.15.6:38214) 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 -- 172.21.15.6:0/2689487990 learned_addr learned my addr 172.21.15.6:0/2689487990 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff809400640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff804102260 0x7ff804073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff809e00640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804101070 0x7ff804074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:57.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 -- 172.21.15.6:0/2689487990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804101070 msgr2=0x7ff804074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804101070 0x7ff804074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 -- 172.21.15.6:0/2689487990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff804102260 msgr2=0x7ff804073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff804102260 0x7ff804073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.744+0000 7ff80a800640 1 -- 172.21.15.6:0/2689487990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4009560 con 0x7ff804102c20 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.745+0000 7ff80a800640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 0x7ff804073ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff80000eb40 tx=0x7ff80000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.745+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800004b20 con 0x7ff804102c20 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.745+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff800004c80 con 0x7ff804102c20 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.745+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800010670 con 0x7ff804102c20 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.745+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8041a1420 con 0x7ff804102c20 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.745+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8041a18b0 con 0x7ff804102c20 2024-02-26T06:26:57.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.746+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff804108390 con 0x7ff804102c20 2024-02-26T06:26:57.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.748+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff800005020 con 0x7ff804102c20 2024-02-26T06:26:57.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.749+0000 7ff7fea00640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7ec0a20b0 0x7ff7ec0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:57.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.750+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff800014070 con 0x7ff804102c20 2024-02-26T06:26:57.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.750+0000 7ff809e00640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7ec0a20b0 0x7ff7ec0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:57.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.750+0000 7ff809e00640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7ec0a20b0 0x7ff7ec0a4570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff8041020c0 tx=0x7ff7f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:57.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.754+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff800097230 con 0x7ff804102c20 2024-02-26T06:26:57.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.956+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff80410ca80 con 0x7ff804102c20 2024-02-26T06:26:57.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.959+0000 7ff7fea00640 1 -- 172.21.15.6:0/2689487990 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7ff80009a8f0 con 0x7ff804102c20 2024-02-26T06:26:57.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:26:57.959 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7ec0a20b0 msgr2=0x7ff7ec0a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7ec0a20b0 0x7ff7ec0a4570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff8041020c0 tx=0x7ff7f8005f90 comp rx=0 tx=0).stop 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 msgr2=0x7ff804073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 0x7ff804073ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff80000eb40 tx=0x7ff80000c2d0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 shutdown_connections 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff804102c20 0x7ff804073ab0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff7ec0a20b0 0x7ff7ec0a4570 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff804102260 0x7ff804073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 --2- 172.21.15.6:0/2689487990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804101070 0x7ff804074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.961+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 >> 172.21.15.6:0/2689487990 conn(0x7ff8040fc5e0 msgr2=0x7ff804105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:57.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.962+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 shutdown_connections 2024-02-26T06:26:57.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:57.962+0000 7ff80b200640 1 -- 172.21.15.6:0/2689487990 wait complete. 2024-02-26T06:26:57.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:26:57.981 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 -- 172.21.15.6:0/3956704958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea8104f60 msgr2=0x7faea810b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 --2- 172.21.15.6:0/3956704958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea8104f60 0x7faea810b7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faea0009900 tx=0x7faea002bc30 comp rx=0 tx=0).stop 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 -- 172.21.15.6:0/3956704958 shutdown_connections 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 --2- 172.21.15.6:0/3956704958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea8104f60 0x7faea810b7f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 --2- 172.21.15.6:0/3956704958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 0x7faea8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 --2- 172.21.15.6:0/3956704958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faea81033b0 0x7faea81037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 -- 172.21.15.6:0/3956704958 >> 172.21.15.6:0/3956704958 conn(0x7faea80fe940 msgr2=0x7faea8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 -- 172.21.15.6:0/3956704958 shutdown_connections 2024-02-26T06:26:58.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.202+0000 7faead200640 1 -- 172.21.15.6:0/3956704958 wait complete. 2024-02-26T06:26:58.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 Processor -- start 2024-02-26T06:26:58.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 -- start start 2024-02-26T06:26:58.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea81033b0 0x7faea8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 0x7faea8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faea8104f60 0x7faea81a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81a0ef0 con 0x7faea8104f60 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81a1060 con 0x7faea81033b0 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.204+0000 7faead200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81a1220 con 0x7faea81045a0 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea81033b0 0x7faea8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea6800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faea8104f60 0x7faea81a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea81033b0 0x7faea8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33030/0 (socket says 172.21.15.6:33030) 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5e00640 1 -- 172.21.15.6:0/710899032 learned_addr learned my addr 172.21.15.6:0/710899032 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 0x7faea8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 -- 172.21.15.6:0/710899032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea81033b0 msgr2=0x7faea8199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea81033b0 0x7faea8199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 -- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faea8104f60 msgr2=0x7faea81a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faea8104f60 0x7faea81a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 -- 172.21.15.6:0/710899032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea0009560 con 0x7faea81045a0 2024-02-26T06:26:58.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.205+0000 7faea5400640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 0x7faea8199940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae9000e9f0 tx=0x7fae9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.206+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000ccc0 con 0x7faea81045a0 2024-02-26T06:26:58.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.206+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9000ce20 con 0x7faea81045a0 2024-02-26T06:26:58.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.206+0000 7faead200640 1 -- 172.21.15.6:0/710899032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea81a1500 con 0x7faea81045a0 2024-02-26T06:26:58.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.206+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90004a00 con 0x7faea81045a0 2024-02-26T06:26:58.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.206+0000 7faead200640 1 -- 172.21.15.6:0/710899032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea81a1a10 con 0x7faea81045a0 2024-02-26T06:26:58.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.207+0000 7faead200640 1 -- 172.21.15.6:0/710899032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea810a670 con 0x7faea81045a0 2024-02-26T06:26:58.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.209+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fae900050b0 con 0x7faea81045a0 2024-02-26T06:26:58.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.211+0000 7fae9ea00640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae8c0a2140 0x7fae8c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.211+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fae90014070 con 0x7faea81045a0 2024-02-26T06:26:58.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.211+0000 7faea5e00640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae8c0a2140 0x7fae8c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.212+0000 7faea5e00640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae8c0a2140 0x7fae8c0a4600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fae98011440 tx=0x7fae98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.215+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae90096b90 con 0x7faea81045a0 2024-02-26T06:26:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:58 smithi090 ceph-mon[35681]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-02-26T06:26:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:58 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2689487990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:26:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:58 smithi006 ceph-mon[31790]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-02-26T06:26:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:58 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2689487990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:26:58.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.423+0000 7faead200640 1 -- 172.21.15.6:0/710899032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faea802ce40 con 0x7faea81045a0 2024-02-26T06:26:58.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.424+0000 7fae9ea00640 1 -- 172.21.15.6:0/710899032 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae9009a250 con 0x7faea81045a0 2024-02-26T06:26:58.423 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 -- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae8c0a2140 msgr2=0x7fae8c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae8c0a2140 0x7fae8c0a4600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fae98011440 tx=0x7fae98009500 comp rx=0 tx=0).stop 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 -- 172.21.15.6:0/710899032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 msgr2=0x7faea8199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 0x7faea8199940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae9000e9f0 tx=0x7fae9000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 -- 172.21.15.6:0/710899032 shutdown_connections 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faea8104f60 0x7faea81a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faea81045a0 0x7faea8199940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fae8c0a2140 0x7fae8c0a4600 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 --2- 172.21.15.6:0/710899032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faea81033b0 0x7faea8199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 -- 172.21.15.6:0/710899032 >> 172.21.15.6:0/710899032 conn(0x7faea80fe940 msgr2=0x7faea8107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 -- 172.21.15.6:0/710899032 shutdown_connections 2024-02-26T06:26:58.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.427+0000 7faead200640 1 -- 172.21.15.6:0/710899032 wait complete. 2024-02-26T06:26:58.447 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:26:58.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:58 smithi167 ceph-mon[36229]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-02-26T06:26:58.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:58 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2689487990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 -- 172.21.15.6:0/4157572293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd4104d90 msgr2=0x7ffbd410b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/4157572293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd4104d90 0x7ffbd410b620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009ca0 tx=0x7ffbcc02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 -- 172.21.15.6:0/4157572293 shutdown_connections 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/4157572293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd4104d90 0x7ffbd410b620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/4157572293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbd41043d0 0x7ffbd4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/4157572293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41031e0 0x7ffbd4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 -- 172.21.15.6:0/4157572293 >> 172.21.15.6:0/4157572293 conn(0x7ffbd40fe750 msgr2=0x7ffbd4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 -- 172.21.15.6:0/4157572293 shutdown_connections 2024-02-26T06:26:58.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.676+0000 7ffbd3e00640 1 -- 172.21.15.6:0/4157572293 wait complete. 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.677+0000 7ffbd3e00640 1 Processor -- start 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 -- start start 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 0x7ffbd4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41043d0 0x7ffbd4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbd4104d90 0x7ffbd41a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41a0d40 con 0x7ffbd41043d0 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41a0eb0 con 0x7ffbd41031e0 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41a1070 con 0x7ffbd4104d90 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd2000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41043d0 0x7ffbd4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 0x7ffbd4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 0x7ffbd4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33048/0 (socket says 172.21.15.6:33048) 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd2a00640 1 -- 172.21.15.6:0/3762217030 learned_addr learned my addr 172.21.15.6:0/3762217030 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.678+0000 7ffbd3400640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbd4104d90 0x7ffbd41a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2a00640 1 -- 172.21.15.6:0/3762217030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbd4104d90 msgr2=0x7ffbd41a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2a00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbd4104d90 0x7ffbd41a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2a00640 1 -- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41043d0 msgr2=0x7ffbd4199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2a00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41043d0 0x7ffbd4199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2a00640 1 -- 172.21.15.6:0/3762217030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc009900 con 0x7ffbd41031e0 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2000640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41043d0 0x7ffbd4199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbd2a00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 0x7ffbd4199250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400e9c0 tx=0x7ffbc400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4009700 con 0x7ffbd41031e0 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.679+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbc400cd80 con 0x7ffbd41031e0 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.680+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd41a1350 con 0x7ffbd41031e0 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.680+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4010430 con 0x7ffbd41031e0 2024-02-26T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.680+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd41a1860 con 0x7ffbd41031e0 2024-02-26T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.681+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd410a500 con 0x7ffbd41031e0 2024-02-26T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.683+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ffbc4003680 con 0x7ffbd41031e0 2024-02-26T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.684+0000 7ffbcb400640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffbb80a20c0 0x7ffbb80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.684+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ffbc4014070 con 0x7ffbd41031e0 2024-02-26T06:26:58.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.684+0000 7ffbd2000640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffbb80a20c0 0x7ffbb80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:58.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.685+0000 7ffbd2000640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffbb80a20c0 0x7ffbb80a4580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffbd419a760 tx=0x7ffbc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.688+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffbc4097040 con 0x7ffbd41031e0 2024-02-26T06:26:58.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.896+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbd410ebf0 con 0x7ffbd41031e0 2024-02-26T06:26:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.897+0000 7ffbcb400640 1 -- 172.21.15.6:0/3762217030 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffbc409a700 con 0x7ffbd41031e0 2024-02-26T06:26:58.897 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:26:58.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffbb80a20c0 msgr2=0x7ffbb80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffbb80a20c0 0x7ffbb80a4580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffbd419a760 tx=0x7ffbc0005f90 comp rx=0 tx=0).stop 2024-02-26T06:26:58.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 msgr2=0x7ffbd4199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 0x7ffbd4199250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400e9c0 tx=0x7ffbc400ee80 comp rx=0 tx=0).stop 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 shutdown_connections 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbd4104d90 0x7ffbd41a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbd41043d0 0x7ffbd4199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ffbb80a20c0 0x7ffbb80a4580 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 --2- 172.21.15.6:0/3762217030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbd41031e0 0x7ffbd4199250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 >> 172.21.15.6:0/3762217030 conn(0x7ffbd40fe750 msgr2=0x7ffbd41076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 shutdown_connections 2024-02-26T06:26:58.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:58.900+0000 7ffbd3e00640 1 -- 172.21.15.6:0/3762217030 wait complete. 2024-02-26T06:26:58.920 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:26:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 -- 172.21.15.6:0/2205209154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 msgr2=0x7f62d81013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 --2- 172.21.15.6:0/2205209154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d81013f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009ca0 tx=0x7f62c802c100 comp rx=0 tx=0).stop 2024-02-26T06:26:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 -- 172.21.15.6:0/2205209154 shutdown_connections 2024-02-26T06:26:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 --2- 172.21.15.6:0/2205209154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 0x7f62d8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 --2- 172.21.15.6:0/2205209154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62d81021c0 0x7f62d8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 --2- 172.21.15.6:0/2205209154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d81013f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.143+0000 7f62d7e00640 1 -- 172.21.15.6:0/2205209154 >> 172.21.15.6:0/2205209154 conn(0x7f62d80fc540 msgr2=0x7f62d80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:59.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.144+0000 7f62d7e00640 1 -- 172.21.15.6:0/2205209154 shutdown_connections 2024-02-26T06:26:59.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.144+0000 7f62d7e00640 1 -- 172.21.15.6:0/2205209154 wait complete. 2024-02-26T06:26:59.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.145+0000 7f62d7e00640 1 Processor -- start 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.145+0000 7f62d7e00640 1 -- start start 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.145+0000 7f62d7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d8074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.145+0000 7f62d7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62d81021c0 0x7f62d8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.145+0000 7f62d7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 0x7f62d8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.145+0000 7f62d7e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8074210 con 0x7f62d81021c0 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8074380 con 0x7f62d8100fd0 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d81a10c0 con 0x7f62d8102b80 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d8074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d8074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33060/0 (socket says 172.21.15.6:33060) 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d6a00640 1 -- 172.21.15.6:0/3874910914 learned_addr learned my addr 172.21.15.6:0/3874910914 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7400640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 0x7f62d8073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62ce000640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62d81021c0 0x7f62d8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7400640 1 -- 172.21.15.6:0/3874910914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 msgr2=0x7f62d8074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7400640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d8074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7400640 1 -- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62d81021c0 msgr2=0x7f62d8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7400640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62d81021c0 0x7f62d8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d7400640 1 -- 172.21.15.6:0/3874910914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c8009900 con 0x7f62d8102b80 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.146+0000 7f62d6a00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.147+0000 7f62d7400640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 0x7f62d8073ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e970 tx=0x7f62d000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.147+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0009620 con 0x7f62d8102b80 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.147+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d81a1370 con 0x7f62d8102b80 2024-02-26T06:26:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.147+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62d000ccc0 con 0x7f62d8102b80 2024-02-26T06:26:59.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.147+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0004a00 con 0x7f62d8102b80 2024-02-26T06:26:59.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.147+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d81a1880 con 0x7f62d8102b80 2024-02-26T06:26:59.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.148+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d81082f0 con 0x7f62d8102b80 2024-02-26T06:26:59.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.150+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f62d0005020 con 0x7f62d8102b80 2024-02-26T06:26:59.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.152+0000 7f62cfe00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62bc0a20c0 0x7f62bc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.152+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f62d0014070 con 0x7f62d8102b80 2024-02-26T06:26:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.152+0000 7f62d6a00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62bc0a20c0 0x7f62bc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.153+0000 7f62d6a00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62bc0a20c0 0x7f62bc0a4580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005e30 tx=0x7f62c8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.156+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62d0096b60 con 0x7f62d8102b80 2024-02-26T06:26:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:59 smithi006 ceph-mon[31790]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/710899032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:26:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:26:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3762217030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:26:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.361+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62d802ce40 con 0x7f62d8102b80 2024-02-26T06:26:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.362+0000 7f62cfe00640 1 -- 172.21.15.6:0/3874910914 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62d009a220 con 0x7f62d8102b80 2024-02-26T06:26:59.361 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62bc0a20c0 msgr2=0x7f62bc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62bc0a20c0 0x7f62bc0a4580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005e30 tx=0x7f62c8004a40 comp rx=0 tx=0).stop 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 msgr2=0x7f62d8073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 0x7f62d8073ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e970 tx=0x7f62d000ee30 comp rx=0 tx=0).stop 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 shutdown_connections 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d8102b80 0x7f62d8073ab0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62d81021c0 0x7f62d8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62bc0a20c0 0x7f62bc0a4580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 --2- 172.21.15.6:0/3874910914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d8100fd0 0x7f62d8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.365+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 >> 172.21.15.6:0/3874910914 conn(0x7f62d80fc540 msgr2=0x7f62d81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.366+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 shutdown_connections 2024-02-26T06:26:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.366+0000 7f62d7e00640 1 -- 172.21.15.6:0/3874910914 wait complete. 2024-02-26T06:26:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:59 smithi090 ceph-mon[35681]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:59 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/710899032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:26:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:26:59 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3762217030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:26:59.385 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:26:59.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:59 smithi167 ceph-mon[36229]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:26:59.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:59 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/710899032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:26:59.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:26:59 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3762217030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:26:59.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 -- 172.21.15.6:0/3782493225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204075140 msgr2=0x7fc204073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 --2- 172.21.15.6:0/3782493225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204075140 0x7fc204073590 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009900 tx=0x7fc1f402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 -- 172.21.15.6:0/3782493225 shutdown_connections 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 --2- 172.21.15.6:0/3782493225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc204108f30 0x7fc20410b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 --2- 172.21.15.6:0/3782493225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc204073b60 0x7fc204073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 --2- 172.21.15.6:0/3782493225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204075140 0x7fc204073590 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 -- 172.21.15.6:0/3782493225 >> 172.21.15.6:0/3782493225 conn(0x7fc2040fc610 msgr2=0x7fc2040fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 -- 172.21.15.6:0/3782493225 shutdown_connections 2024-02-26T06:26:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.613+0000 7fc20a200640 1 -- 172.21.15.6:0/3782493225 wait complete. 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.614+0000 7fc20a200640 1 Processor -- start 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.614+0000 7fc20a200640 1 -- start start 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc20a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc204073b60 0x7fc2041992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc20a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc204075140 0x7fc204199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc20a200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 0x7fc2041a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc20a200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2041a0de0 con 0x7fc204108f30 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc20a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2041a0f50 con 0x7fc204075140 2024-02-26T06:26:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc20a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2041a1110 con 0x7fc204073b60 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 0x7fc2041a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 0x7fc2041a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38300/0 (socket says 172.21.15.6:38300) 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 -- 172.21.15.6:0/891624259 learned_addr learned my addr 172.21.15.6:0/891624259 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc208e00640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc204073b60 0x7fc2041992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc1fbe00640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc204075140 0x7fc204199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 -- 172.21.15.6:0/891624259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc204073b60 msgr2=0x7fc2041992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc204073b60 0x7fc2041992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 -- 172.21.15.6:0/891624259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc204075140 msgr2=0x7fc204199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc204075140 0x7fc204199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.615+0000 7fc209800640 1 -- 172.21.15.6:0/891624259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f4009560 con 0x7fc204108f30 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.616+0000 7fc209800640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 0x7fc2041a08a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00eaa0 tx=0x7fc1fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.616+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00cdb0 con 0x7fc204108f30 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.616+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2041a13f0 con 0x7fc204108f30 2024-02-26T06:26:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.616+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc005020 con 0x7fc204108f30 2024-02-26T06:26:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.616+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0116d0 con 0x7fc204108f30 2024-02-26T06:26:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.616+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2041a1900 con 0x7fc204108f30 2024-02-26T06:26:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.617+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc204107cf0 con 0x7fc204108f30 2024-02-26T06:26:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.620+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc1fc005490 con 0x7fc204108f30 2024-02-26T06:26:59.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.621+0000 7fc202000640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1e80a20b0 0x7fc1e80a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:26:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.622+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc1fc0cdde0 con 0x7fc204108f30 2024-02-26T06:26:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.622+0000 7fc208e00640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1e80a20b0 0x7fc1e80a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:26:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.622+0000 7fc208e00640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1e80a20b0 0x7fc1e80a4570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000c00 tx=0x7fc1f402c870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:26:59.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.626+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1fc096df0 con 0x7fc204108f30 2024-02-26T06:26:59.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.871+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc20402cda0 con 0x7fc204108f30 2024-02-26T06:26:59.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.873+0000 7fc202000640 1 -- 172.21.15.6:0/891624259 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11356 (secure 0 0 0) 0x7fc1fc01f020 con 0x7fc204108f30 2024-02-26T06:26:59.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:26:59.872 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:25:24.777181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:26:59.873 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:26:59.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1e80a20b0 msgr2=0x7fc1e80a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1e80a20b0 0x7fc1e80a4570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000c00 tx=0x7fc1f402c870 comp rx=0 tx=0).stop 2024-02-26T06:26:59.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 msgr2=0x7fc2041a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:26:59.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 0x7fc2041a08a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00eaa0 tx=0x7fc1fc00ef60 comp rx=0 tx=0).stop 2024-02-26T06:26:59.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 shutdown_connections 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc204108f30 0x7fc2041a08a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc1e80a20b0 0x7fc1e80a4570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc204075140 0x7fc204199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 --2- 172.21.15.6:0/891624259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc204073b60 0x7fc2041992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 >> 172.21.15.6:0/891624259 conn(0x7fc2040fc610 msgr2=0x7fc204104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 shutdown_connections 2024-02-26T06:26:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:26:59.876+0000 7fc20a200640 1 -- 172.21.15.6:0/891624259 wait complete. 2024-02-26T06:26:59.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:00.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 -- 172.21.15.6:0/690876243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac241032a0 msgr2=0x7fac241036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 --2- 172.21.15.6:0/690876243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac241032a0 0x7fac241036c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fac14009900 tx=0x7fac1402bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:00.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 -- 172.21.15.6:0/690876243 shutdown_connections 2024-02-26T06:27:00.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 --2- 172.21.15.6:0/690876243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac24104e50 0x7fac2410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 --2- 172.21.15.6:0/690876243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104490 0x7fac24104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 --2- 172.21.15.6:0/690876243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac241032a0 0x7fac241036c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 -- 172.21.15.6:0/690876243 >> 172.21.15.6:0/690876243 conn(0x7fac240fe830 msgr2=0x7fac24100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:00.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 -- 172.21.15.6:0/690876243 shutdown_connections 2024-02-26T06:27:00.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.126+0000 7fac23200640 1 -- 172.21.15.6:0/690876243 wait complete. 2024-02-26T06:27:00.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.127+0000 7fac23200640 1 Processor -- start 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.127+0000 7fac23200640 1 -- start start 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac23200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 0x7fac24199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac23200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac24104490 0x7fac24199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac23200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104e50 0x7fac241a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac23200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241a0e00 con 0x7fac241032a0 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac23200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241a0f70 con 0x7fac24104490 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac23200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241a1130 con 0x7fac24104e50 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac21e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 0x7fac24199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac21e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 0x7fac24199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38312/0 (socket says 172.21.15.6:38312) 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac21e00640 1 -- 172.21.15.6:0/3554680584 learned_addr learned my addr 172.21.15.6:0/3554680584 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac22800640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104e50 0x7fac241a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.128+0000 7fac21400640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac24104490 0x7fac24199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21e00640 1 -- 172.21.15.6:0/3554680584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104e50 msgr2=0x7fac241a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21e00640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104e50 0x7fac241a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21e00640 1 -- 172.21.15.6:0/3554680584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac24104490 msgr2=0x7fac24199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21e00640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac24104490 0x7fac24199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21e00640 1 -- 172.21.15.6:0/3554680584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009560 con 0x7fac241032a0 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac22800640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104e50 0x7fac241a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21400640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac24104490 0x7fac24199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac21e00640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 0x7fac24199310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac14000c00 tx=0x7fac14005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140043d0 con 0x7fac241032a0 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac140052d0 con 0x7fac241032a0 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac241a13b0 con 0x7fac241032a0 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.129+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14003da0 con 0x7fac241032a0 2024-02-26T06:27:00.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.130+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac241a18c0 con 0x7fac241032a0 2024-02-26T06:27:00.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.131+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2410a5c0 con 0x7fac241032a0 2024-02-26T06:27:00.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.133+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fac14005440 con 0x7fac241032a0 2024-02-26T06:27:00.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.135+0000 7fac1aa00640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac080a20b0 0x7fac080a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.135+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fac140f0440 con 0x7fac241032a0 2024-02-26T06:27:00.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.135+0000 7fac21400640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac080a20b0 0x7fac080a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.136+0000 7fac21400640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac080a20b0 0x7fac080a4570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fac2419a820 tx=0x7fac10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:00.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.139+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac140b9520 con 0x7fac241032a0 2024-02-26T06:27:00.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:00 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3874910914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:00.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:00 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/891624259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:00.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.343+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac2402cda0 con 0x7fac241032a0 2024-02-26T06:27:00.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.345+0000 7fac1aa00640 1 -- 172.21.15.6:0/3554680584 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7fac14043020 con 0x7fac241032a0 2024-02-26T06:27:00.345 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:00.345 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:00.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac080a20b0 msgr2=0x7fac080a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac080a20b0 0x7fac080a4570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fac2419a820 tx=0x7fac10005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:00.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 msgr2=0x7fac24199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 0x7fac24199310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac14000c00 tx=0x7fac14005e80 comp rx=0 tx=0).stop 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 shutdown_connections 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac24104e50 0x7fac241a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fac080a20b0 0x7fac080a4570 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac24104490 0x7fac24199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 --2- 172.21.15.6:0/3554680584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac241032a0 0x7fac24199310 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 >> 172.21.15.6:0/3554680584 conn(0x7fac240fe830 msgr2=0x7fac24107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 shutdown_connections 2024-02-26T06:27:00.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.348+0000 7fac23200640 1 -- 172.21.15.6:0/3554680584 wait complete. 2024-02-26T06:27:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:00 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3874910914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:00 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/891624259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:00.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:27:00.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:27:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:00 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3874910914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:00 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/891624259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.598+0000 7fc488a00640 1 -- 172.21.15.6:0/2709188360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4841021b0 msgr2=0x7fc484104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.598+0000 7fc488a00640 1 --2- 172.21.15.6:0/2709188360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4841021b0 0x7fc484104620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009ca0 tx=0x7fc46c02c100 comp rx=0 tx=0).stop 2024-02-26T06:27:00.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 -- 172.21.15.6:0/2709188360 shutdown_connections 2024-02-26T06:27:00.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 --2- 172.21.15.6:0/2709188360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc484104b60 0x7fc484104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 --2- 172.21.15.6:0/2709188360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4841021b0 0x7fc484104620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 --2- 172.21.15.6:0/2709188360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4840ff840 0x7fc484101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 -- 172.21.15.6:0/2709188360 >> 172.21.15.6:0/2709188360 conn(0x7fc4840fb460 msgr2=0x7fc4840fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:00.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 -- 172.21.15.6:0/2709188360 shutdown_connections 2024-02-26T06:27:00.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.599+0000 7fc488a00640 1 -- 172.21.15.6:0/2709188360 wait complete. 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.600+0000 7fc488a00640 1 Processor -- start 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.600+0000 7fc488a00640 1 -- start start 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc488a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4840ff840 0x7fc48419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc488a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4841021b0 0x7fc48419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc488a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 0x7fc4841a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc488a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841a5150 con 0x7fc484104b60 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc488a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841a52c0 con 0x7fc4841021b0 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc488a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841a5480 con 0x7fc4840ff840 2024-02-26T06:27:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 0x7fc4841a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 0x7fc4841a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38324/0 (socket says 172.21.15.6:38324) 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 -- 172.21.15.6:0/142505209 learned_addr learned my addr 172.21.15.6:0/142505209 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc482800640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4840ff840 0x7fc48419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc481e00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4841021b0 0x7fc48419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 -- 172.21.15.6:0/142505209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4840ff840 msgr2=0x7fc48419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4840ff840 0x7fc48419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 -- 172.21.15.6:0/142505209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4841021b0 msgr2=0x7fc48419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4841021b0 0x7fc48419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc483200640 1 -- 172.21.15.6:0/142505209 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c009900 con 0x7fc484104b60 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.601+0000 7fc482800640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4840ff840 0x7fc48419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.602+0000 7fc483200640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 0x7fc4841a4c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00e970 tx=0x7fc47c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.602+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c009720 con 0x7fc484104b60 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.602+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4841a5760 con 0x7fc484104b60 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.602+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc47c005500 con 0x7fc484104b60 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.602+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c010430 con 0x7fc484104b60 2024-02-26T06:27:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.602+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4841a5c90 con 0x7fc484104b60 2024-02-26T06:27:00.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.603+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4840fcf50 con 0x7fc484104b60 2024-02-26T06:27:00.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.605+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc47c004ff0 con 0x7fc484104b60 2024-02-26T06:27:00.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.608+0000 7fc47b400640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4640a2140 0x7fc4640a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:00.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.608+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc47c014070 con 0x7fc484104b60 2024-02-26T06:27:00.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.608+0000 7fc482800640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4640a2140 0x7fc4640a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:00.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.609+0000 7fc482800640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4640a2140 0x7fc4640a4600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc474009fc0 tx=0x7fc474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:00.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.612+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc47c0d1030 con 0x7fc484104b60 2024-02-26T06:27:00.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.818+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc48402ce40 con 0x7fc484104b60 2024-02-26T06:27:00.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.819+0000 7fc47b400640 1 -- 172.21.15.6:0/142505209 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc47c096f30 con 0x7fc484104b60 2024-02-26T06:27:00.818 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:27:00.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.821+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4640a2140 msgr2=0x7fc4640a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.821+0000 7fc488a00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4640a2140 0x7fc4640a4600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc474009fc0 tx=0x7fc474005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:00.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 msgr2=0x7fc4841a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:00.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 0x7fc4841a4c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00e970 tx=0x7fc47c00ed70 comp rx=0 tx=0).stop 2024-02-26T06:27:00.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 shutdown_connections 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc484104b60 0x7fc4841a4c10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc4640a2140 0x7fc4640a4600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4841021b0 0x7fc48419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 --2- 172.21.15.6:0/142505209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4840ff840 0x7fc48419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 >> 172.21.15.6:0/142505209 conn(0x7fc4840fb460 msgr2=0x7fc484100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 shutdown_connections 2024-02-26T06:27:00.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:00.822+0000 7fc488a00640 1 -- 172.21.15.6:0/142505209 wait complete. 2024-02-26T06:27:00.839 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.069+0000 7f891ca00640 1 -- 172.21.15.6:0/359773031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 msgr2=0x7f8918103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.069+0000 7f891ca00640 1 --2- 172.21.15.6:0/359773031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f8918103e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 -- 172.21.15.6:0/359773031 shutdown_connections 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 --2- 172.21.15.6:0/359773031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918106cb0 0x7f8918109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 --2- 172.21.15.6:0/359773031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918104380 0x7f8918106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 --2- 172.21.15.6:0/359773031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f8918103e40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 -- 172.21.15.6:0/359773031 >> 172.21.15.6:0/359773031 conn(0x7f89180fb460 msgr2=0x7f89180fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 -- 172.21.15.6:0/359773031 shutdown_connections 2024-02-26T06:27:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.070+0000 7f891ca00640 1 -- 172.21.15.6:0/359773031 wait complete. 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 Processor -- start 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 -- start start 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f89181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918104380 0x7f8918199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918106cb0 0x7f89181a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181a0dc0 con 0x7f8918101a50 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181a0f30 con 0x7f8918104380 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.071+0000 7f891ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181a10f0 con 0x7f8918106cb0 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f89181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f890fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918104380 0x7f8918199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f89181992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38344/0 (socket says 172.21.15.6:38344) 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 -- 172.21.15.6:0/1014056262 learned_addr learned my addr 172.21.15.6:0/1014056262 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:01.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8917200640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918106cb0 0x7f89181a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 -- 172.21.15.6:0/1014056262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918106cb0 msgr2=0x7f89181a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918106cb0 0x7f89181a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 -- 172.21.15.6:0/1014056262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918104380 msgr2=0x7f8918199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918104380 0x7f8918199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8916800640 1 -- 172.21.15.6:0/1014056262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910009840 con 0x7f8918101a50 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f890fe00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918104380 0x7f8918199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.072+0000 7f8917200640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918106cb0 0x7f89181a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.073+0000 7f8916800640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f89181992d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f890802c6a0 tx=0x7f89080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.073+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080044c0 con 0x7f8918101a50 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.073+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89080052d0 con 0x7f8918101a50 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.073+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908003e10 con 0x7f8918101a50 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.073+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8908009560 con 0x7f8918101a50 2024-02-26T06:27:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.073+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89181a16a0 con 0x7f8918101a50 2024-02-26T06:27:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.074+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8918108800 con 0x7f8918101a50 2024-02-26T06:27:01.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.076+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8908005440 con 0x7f8918101a50 2024-02-26T06:27:01.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.078+0000 7f890f400640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f88fc0a20c0 0x7f88fc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.078+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f89080f0500 con 0x7f8918101a50 2024-02-26T06:27:01.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.078+0000 7f890fe00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f88fc0a20c0 0x7f88fc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.079+0000 7f890fe00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f88fc0a20c0 0x7f88fc0a4580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f891819a7e0 tx=0x7f8900005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.082+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89080b9580 con 0x7f8918101a50 2024-02-26T06:27:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:01 smithi006 ceph-mon[31790]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3554680584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:01.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/142505209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.289+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f891802ce40 con 0x7f8918101a50 2024-02-26T06:27:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.290+0000 7f890f400640 1 -- 172.21.15.6:0/1014056262 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89080430a0 con 0x7f8918101a50 2024-02-26T06:27:01.290 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.293+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f88fc0a20c0 msgr2=0x7f88fc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.293+0000 7f891ca00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f88fc0a20c0 0x7f88fc0a4580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f891819a7e0 tx=0x7f8900005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.293+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 msgr2=0x7f89181992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.293+0000 7f891ca00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f89181992d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f890802c6a0 tx=0x7f89080040c0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 shutdown_connections 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8918106cb0 0x7f89181a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f88fc0a20c0 0x7f88fc0a4580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8918104380 0x7f8918199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 --2- 172.21.15.6:0/1014056262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8918101a50 0x7f89181992d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 >> 172.21.15.6:0/1014056262 conn(0x7f89180fb460 msgr2=0x7f8918101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 shutdown_connections 2024-02-26T06:27:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.294+0000 7f891ca00640 1 -- 172.21.15.6:0/1014056262 wait complete. 2024-02-26T06:27:01.311 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:27:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:01 smithi090 ceph-mon[35681]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:01 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3554680584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:01 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/142505209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:01.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:01 smithi167 ceph-mon[36229]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:01.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:01 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3554680584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:01.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:01 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/142505209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 -- 172.21.15.6:0/114733216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08041031e0 msgr2=0x7f0804103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 --2- 172.21.15.6:0/114733216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08041031e0 0x7f0804103600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009900 tx=0x7f07f802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 -- 172.21.15.6:0/114733216 shutdown_connections 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 --2- 172.21.15.6:0/114733216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0804104d90 0x7f080410b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 --2- 172.21.15.6:0/114733216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 --2- 172.21.15.6:0/114733216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08041031e0 0x7f0804103600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.538+0000 7f080ae00640 1 -- 172.21.15.6:0/114733216 >> 172.21.15.6:0/114733216 conn(0x7f08040fe750 msgr2=0x7f0804100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.539+0000 7f080ae00640 1 -- 172.21.15.6:0/114733216 shutdown_connections 2024-02-26T06:27:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.539+0000 7f080ae00640 1 -- 172.21.15.6:0/114733216 wait complete. 2024-02-26T06:27:01.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 Processor -- start 2024-02-26T06:27:01.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 -- start start 2024-02-26T06:27:01.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08041031e0 0x7f08041992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0804104d90 0x7f08041a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08041a0dc0 con 0x7f08041043d0 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08041a0f30 con 0x7f0804104d90 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08041a10f0 con 0x7f08041031e0 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f0809000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f0809000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38362/0 (socket says 172.21.15.6:38362) 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f0809000640 1 -- 172.21.15.6:0/2428678803 learned_addr learned my addr 172.21.15.6:0/2428678803 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f0809a00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08041031e0 0x7f08041992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.540+0000 7f080a400640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0804104d90 0x7f08041a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0809000640 1 -- 172.21.15.6:0/2428678803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08041031e0 msgr2=0x7f08041992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0809000640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08041031e0 0x7f08041992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0809000640 1 -- 172.21.15.6:0/2428678803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0804104d90 msgr2=0x7f08041a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0809000640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0804104d90 0x7f08041a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0809000640 1 -- 172.21.15.6:0/2428678803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009560 con 0x7f08041043d0 2024-02-26T06:27:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0809000640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804199810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f07f400e9f0 tx=0x7f07f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4009700 con 0x7f08041043d0 2024-02-26T06:27:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07f400cd80 con 0x7f08041043d0 2024-02-26T06:27:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018920 con 0x7f08041043d0 2024-02-26T06:27:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.541+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08041a13d0 con 0x7f08041043d0 2024-02-26T06:27:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.542+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08041a18e0 con 0x7f08041043d0 2024-02-26T06:27:01.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.543+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080410a500 con 0x7f08041043d0 2024-02-26T06:27:01.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.545+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f07f4018a80 con 0x7f08041043d0 2024-02-26T06:27:01.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.547+0000 7f0802000640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07ec0a20c0 0x7f07ec0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:01.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.547+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f07f4014070 con 0x7f08041043d0 2024-02-26T06:27:01.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.547+0000 7f0809a00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07ec0a20c0 0x7f07ec0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:01.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.548+0000 7f0809a00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07ec0a20c0 0x7f07ec0a4580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07f802c240 tx=0x7f07f802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:01.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.552+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07f4052d20 con 0x7f08041043d0 2024-02-26T06:27:01.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.756+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f080402ce40 con 0x7f08041043d0 2024-02-26T06:27:01.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.757+0000 7f0802000640 1 -- 172.21.15.6:0/2428678803 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07f4097090 con 0x7f08041043d0 2024-02-26T06:27:01.757 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:27:01.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07ec0a20c0 msgr2=0x7f07ec0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07ec0a20c0 0x7f07ec0a4580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07f802c240 tx=0x7f07f802c7e0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 msgr2=0x7f0804199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804199810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f07f400e9f0 tx=0x7f07f400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 shutdown_connections 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f07ec0a20c0 0x7f07ec0a4580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0804104d90 0x7f08041a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08041043d0 0x7f0804199810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 --2- 172.21.15.6:0/2428678803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08041031e0 0x7f08041992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:01.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 >> 172.21.15.6:0/2428678803 conn(0x7f08040fe750 msgr2=0x7f08041076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:01.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.760+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 shutdown_connections 2024-02-26T06:27:01.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:01.761+0000 7f080ae00640 1 -- 172.21.15.6:0/2428678803 wait complete. 2024-02-26T06:27:01.780 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 -- 172.21.15.6:0/1530588950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0101080 msgr2=0x7fc0f01014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 --2- 172.21.15.6:0/1530588950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0101080 0x7fc0f01014a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009900 tx=0x7fc0e402bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 -- 172.21.15.6:0/1530588950 shutdown_connections 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 --2- 172.21.15.6:0/1530588950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0102c30 0x7fc0f01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 --2- 172.21.15.6:0/1530588950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0102270 0x7fc0f01026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 --2- 172.21.15.6:0/1530588950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0101080 0x7fc0f01014a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.005+0000 7fc0f6800640 1 -- 172.21.15.6:0/1530588950 >> 172.21.15.6:0/1530588950 conn(0x7fc0f00fc610 msgr2=0x7fc0f00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.006+0000 7fc0f6800640 1 -- 172.21.15.6:0/1530588950 shutdown_connections 2024-02-26T06:27:02.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.006+0000 7fc0f6800640 1 -- 172.21.15.6:0/1530588950 wait complete. 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 Processor -- start 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 -- start start 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 0x7fc0f01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0102270 0x7fc0f01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0102c30 0x7fc0f01a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01a0e90 con 0x7fc0f0101080 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01a1000 con 0x7fc0f0102c30 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01a11c0 con 0x7fc0f0102270 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f5400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 0x7fc0f01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f5400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 0x7fc0f01993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38378/0 (socket says 172.21.15.6:38378) 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f5400640 1 -- 172.21.15.6:0/336842230 learned_addr learned my addr 172.21.15.6:0/336842230 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f4a00640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0102270 0x7fc0f01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.007+0000 7fc0f5e00640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0102c30 0x7fc0f01a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f5400640 1 -- 172.21.15.6:0/336842230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0102270 msgr2=0x7fc0f01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f5400640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0102270 0x7fc0f01998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f5400640 1 -- 172.21.15.6:0/336842230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0102c30 msgr2=0x7fc0f01a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f5400640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0102c30 0x7fc0f01a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f5400640 1 -- 172.21.15.6:0/336842230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e8009840 con 0x7fc0f0101080 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f4a00640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0102270 0x7fc0f01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:02.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0f5400640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 0x7fc0f01993a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005ab0 tx=0x7fc0e4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:02.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40043d0 con 0x7fc0f0101080 2024-02-26T06:27:02.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.008+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0e40052d0 con 0x7fc0f0101080 2024-02-26T06:27:02.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.009+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4003cd0 con 0x7fc0f0101080 2024-02-26T06:27:02.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.009+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e4009560 con 0x7fc0f0101080 2024-02-26T06:27:02.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.009+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f01a1810 con 0x7fc0f0101080 2024-02-26T06:27:02.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.010+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f010ca90 con 0x7fc0f0101080 2024-02-26T06:27:02.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.012+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc0e4005440 con 0x7fc0f0101080 2024-02-26T06:27:02.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.014+0000 7fc0ee000640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc0d80a2130 0x7fc0d80a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.014+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc0e40efd90 con 0x7fc0f0101080 2024-02-26T06:27:02.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.014+0000 7fc0f4a00640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc0d80a2130 0x7fc0d80a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.015+0000 7fc0f4a00640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc0d80a2130 0x7fc0d80a45f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc0f019a8b0 tx=0x7fc0dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:02.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.018+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc0e40b8cf0 con 0x7fc0f0101080 2024-02-26T06:27:02.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.268+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0f010cc70 con 0x7fc0f0101080 2024-02-26T06:27:02.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.270+0000 7fc0ee000640 1 -- 172.21.15.6:0/336842230 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11356 (secure 0 0 0) 0x7fc0e40bc3b0 con 0x7fc0f0101080 2024-02-26T06:27:02.269 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:02.269 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:25:24.777181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:02.269 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:27:02.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.272+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc0d80a2130 msgr2=0x7fc0d80a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.272+0000 7fc0f6800640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc0d80a2130 0x7fc0d80a45f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc0f019a8b0 tx=0x7fc0dc005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:02.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 msgr2=0x7fc0f01993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 0x7fc0f01993a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005ab0 tx=0x7fc0e4005e50 comp rx=0 tx=0).stop 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 shutdown_connections 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fc0d80a2130 0x7fc0d80a45f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0102c30 0x7fc0f01a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f0102270 0x7fc0f01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 --2- 172.21.15.6:0/336842230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0f0101080 0x7fc0f01993a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 >> 172.21.15.6:0/336842230 conn(0x7fc0f00fc610 msgr2=0x7fc0f00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 shutdown_connections 2024-02-26T06:27:02.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.273+0000 7fc0f6800640 1 -- 172.21.15.6:0/336842230 wait complete. 2024-02-26T06:27:02.292 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:02 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1014056262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:02 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2428678803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:02.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:02 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1014056262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:02.369 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:02 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2428678803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:02.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:02 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1014056262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:02.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:02 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2428678803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 -- 172.21.15.6:0/2356101538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 msgr2=0x7f8d5010b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/2356101538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d5010b6e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009ca0 tx=0x7f8d4802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 -- 172.21.15.6:0/2356101538 shutdown_connections 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/2356101538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d5010b6e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/2356101538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50104490 0x7f8d50104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/2356101538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d501032a0 0x7f8d501036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 -- 172.21.15.6:0/2356101538 >> 172.21.15.6:0/2356101538 conn(0x7f8d500fe830 msgr2=0x7f8d50100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:02.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 -- 172.21.15.6:0/2356101538 shutdown_connections 2024-02-26T06:27:02.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.507+0000 7f8d4fe00640 1 -- 172.21.15.6:0/2356101538 wait complete. 2024-02-26T06:27:02.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.508+0000 7f8d4fe00640 1 Processor -- start 2024-02-26T06:27:02.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 -- start start 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d501032a0 0x7f8d50199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d50104490 0x7f8d501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d501a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501a0d50 con 0x7f8d50104e50 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501a0ec0 con 0x7f8d50104490 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501a1080 con 0x7f8d501032a0 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d501a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d501a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38396/0 (socket says 172.21.15.6:38396) 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d501032a0 0x7f8d50199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d4f400640 1 -- 172.21.15.6:0/3029290891 learned_addr learned my addr 172.21.15.6:0/3029290891 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.509+0000 7f8d47e00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d50104490 0x7f8d501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4ea00640 1 -- 172.21.15.6:0/3029290891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d50104490 msgr2=0x7f8d501997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4ea00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d50104490 0x7f8d501997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4ea00640 1 -- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 msgr2=0x7f8d501a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4ea00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d501a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4ea00640 1 -- 172.21.15.6:0/3029290891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40009490 con 0x7f8d501032a0 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4f400640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d501a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:02.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4ea00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d501032a0 0x7f8d50199260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000e9c0 tx=0x7f8d4000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:02.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4000cd80 con 0x7f8d501032a0 2024-02-26T06:27:02.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d48009900 con 0x7f8d501032a0 2024-02-26T06:27:02.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d40010910 con 0x7f8d501032a0 2024-02-26T06:27:02.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.510+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40018900 con 0x7f8d501032a0 2024-02-26T06:27:02.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.511+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d501a16e0 con 0x7f8d501032a0 2024-02-26T06:27:02.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.512+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5010a560 con 0x7f8d501032a0 2024-02-26T06:27:02.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.514+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8d40010a80 con 0x7f8d501032a0 2024-02-26T06:27:02.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.515+0000 7f8d47400640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8d340a20b0 0x7f8d340a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.515+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8d40014070 con 0x7f8d501032a0 2024-02-26T06:27:02.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.515+0000 7f8d47e00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8d340a20b0 0x7f8d340a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.516+0000 7f8d47e00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8d340a20b0 0x7f8d340a4570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d5019a770 tx=0x7f8d38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:02.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.519+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d400970c0 con 0x7f8d501032a0 2024-02-26T06:27:02.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.726+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d5002cda0 con 0x7f8d501032a0 2024-02-26T06:27:02.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.728+0000 7f8d47400640 1 -- 172.21.15.6:0/3029290891 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f8d4009a780 con 0x7f8d501032a0 2024-02-26T06:27:02.729 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:02.729 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:02.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8d340a20b0 msgr2=0x7f8d340a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8d340a20b0 0x7f8d340a4570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d5019a770 tx=0x7f8d38005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:02.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d501032a0 msgr2=0x7f8d50199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d501032a0 0x7f8d50199260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000e9c0 tx=0x7f8d4000ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 shutdown_connections 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d50104e50 0x7f8d501a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8d340a20b0 0x7f8d340a4570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d50104490 0x7f8d501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 --2- 172.21.15.6:0/3029290891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d501032a0 0x7f8d50199260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 >> 172.21.15.6:0/3029290891 conn(0x7f8d500fe830 msgr2=0x7f8d50107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 shutdown_connections 2024-02-26T06:27:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.732+0000 7f8d4fe00640 1 -- 172.21.15.6:0/3029290891 wait complete. 2024-02-26T06:27:02.751 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:02.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 -- 172.21.15.6:0/935712518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 msgr2=0x7f6260101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 --2- 172.21.15.6:0/935712518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f6260101080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6258009ca0 tx=0x7f625802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 -- 172.21.15.6:0/935712518 shutdown_connections 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 --2- 172.21.15.6:0/935712518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f6260101080 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 --2- 172.21.15.6:0/935712518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6260100260 0x7f62601006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 --2- 172.21.15.6:0/935712518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62600ff900 0x7f62600ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 -- 172.21.15.6:0/935712518 >> 172.21.15.6:0/935712518 conn(0x7f62600fb500 msgr2=0x7f62600fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 -- 172.21.15.6:0/935712518 shutdown_connections 2024-02-26T06:27:02.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.972+0000 7f625fe00640 1 -- 172.21.15.6:0/935712518 wait complete. 2024-02-26T06:27:02.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 Processor -- start 2024-02-26T06:27:02.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 -- start start 2024-02-26T06:27:02.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62600ff900 0x7f6260199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6260100260 0x7f6260199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f62601a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62601a0e30 con 0x7f6260100c00 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62601a0fa0 con 0x7f6260100260 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.973+0000 7f625fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62601a1160 con 0x7f62600ff900 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f62601a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f62601a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38426/0 (socket says 172.21.15.6:38426) 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 -- 172.21.15.6:0/764672926 learned_addr learned my addr 172.21.15.6:0/764672926 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62600ff900 0x7f6260199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625e000640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6260100260 0x7f6260199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 -- 172.21.15.6:0/764672926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62600ff900 msgr2=0x7f6260199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62600ff900 0x7f6260199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 -- 172.21.15.6:0/764672926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6260100260 msgr2=0x7f6260199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6260100260 0x7f6260199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 -- 172.21.15.6:0/764672926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6250009490 con 0x7f6260100c00 2024-02-26T06:27:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625e000640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6260100260 0x7f6260199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.974+0000 7f625f400640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f62601a08f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6258037040 tx=0x7f6258032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.975+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580329b0 con 0x7f6260100c00 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.975+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6258034de0 con 0x7f6260100c00 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.975+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580048a0 con 0x7f6260100c00 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.975+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6258009900 con 0x7f6260100c00 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.975+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62601a1750 con 0x7f6260100c00 2024-02-26T06:27:02.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.976+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626010a650 con 0x7f6260100c00 2024-02-26T06:27:02.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.978+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f625800a040 con 0x7f6260100c00 2024-02-26T06:27:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.979+0000 7f6257400640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62440a2130 0x7f62440a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.979+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f62580f06d0 con 0x7f6260100c00 2024-02-26T06:27:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.980+0000 7f625ea00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62440a2130 0x7f62440a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.980+0000 7f625ea00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62440a2130 0x7f62440a45f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6250009d60 tx=0x7f6250005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:02.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:02.984+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62580b9630 con 0x7f6260100c00 2024-02-26T06:27:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.228+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62601a1320 con 0x7f6260100c00 2024-02-26T06:27:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.229+0000 7f6257400640 1 -- 172.21.15.6:0/764672926 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f62580bccf0 con 0x7f6260100c00 2024-02-26T06:27:03.230 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:03.230 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:25:24.777181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:27:03.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62440a2130 msgr2=0x7f62440a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62440a2130 0x7f62440a45f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6250009d60 tx=0x7f6250005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 msgr2=0x7f62601a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f62601a08f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6258037040 tx=0x7f6258032f80 comp rx=0 tx=0).stop 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 shutdown_connections 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6260100c00 0x7f62601a08f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f62440a2130 0x7f62440a45f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6260100260 0x7f6260199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 --2- 172.21.15.6:0/764672926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62600ff900 0x7f6260199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.232+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 >> 172.21.15.6:0/764672926 conn(0x7f62600fb500 msgr2=0x7f62600fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.233+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 shutdown_connections 2024-02-26T06:27:03.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.233+0000 7f625fe00640 1 -- 172.21.15.6:0/764672926 wait complete. 2024-02-26T06:27:03.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T06:25:17.657061+0000', 'modified': '2024-02-26T06:25:24.777181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24587, 'mds_1': 24569, 'mds_2': 24544, 'mds_3': 24530, 'mds_4': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/66780761', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 66780761}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 66780761}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6833/707593181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6832', 'nonce': 707593181}, {'type': 'v1', 'addr': '172.21.15.90:6833', 'nonce': 707593181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.6:6837/234881158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 234881158}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 234881158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/679395830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 679395830}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 679395830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/2489701568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2489701568}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2489701568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.167:6837/30308127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 30308127}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 30308127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.90:6837/3613516978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 3613516978}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 3613516978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.167:6839/3988004767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3988004767}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3988004767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.6:6841/3799445989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6840', 'nonce': 3799445989}, {'type': 'v1', 'addr': '172.21.15.6:6841', 'nonce': 3799445989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.90:6839/137401855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 137401855}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 137401855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T06:27:03.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-02-26T06:27:03.253 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-26T06:27:03.331 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:03 smithi006 ceph-mon[31790]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:03.331 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:03 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/336842230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:03.331 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:03 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3029290891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:03 smithi090 ceph-mon[35681]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:03 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/336842230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:03 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3029290891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 -- 172.21.15.6:0/688723870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 msgr2=0x7f8fa81037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 --2- 172.21.15.6:0/688723870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa81037d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009900 tx=0x7f8f9802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 -- 172.21.15.6:0/688723870 shutdown_connections 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 --2- 172.21.15.6:0/688723870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 0x7f8fa810b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 --2- 172.21.15.6:0/688723870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 --2- 172.21.15.6:0/688723870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa81037d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.474+0000 7f8faee00640 1 -- 172.21.15.6:0/688723870 >> 172.21.15.6:0/688723870 conn(0x7f8fa80fe940 msgr2=0x7f8fa8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.475+0000 7f8faee00640 1 -- 172.21.15.6:0/688723870 shutdown_connections 2024-02-26T06:27:03.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.475+0000 7f8faee00640 1 -- 172.21.15.6:0/688723870 wait complete. 2024-02-26T06:27:03.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.476+0000 7f8faee00640 1 Processor -- start 2024-02-26T06:27:03.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.476+0000 7f8faee00640 1 -- start start 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8faee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8faee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8faee00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 0x7f8fa81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8faee00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81a0f80 con 0x7f8fa8104f60 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8faee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81a10f0 con 0x7f8fa81045a0 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8faee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81a12b0 con 0x7f8fa81033b0 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8face00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 0x7f8fa81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8fa7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa81999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33226/0 (socket says 172.21.15.6:33226) 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.477+0000 7f8f9f400640 1 -- 172.21.15.6:0/2445354572 learned_addr learned my addr 172.21.15.6:0/2445354572 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa7e00640 1 -- 172.21.15.6:0/2445354572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 msgr2=0x7f8fa81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa7e00640 1 -- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 msgr2=0x7f8fa81a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 0x7f8fa81a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa7e00640 1 -- 172.21.15.6:0/2445354572 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f98009560 con 0x7f8fa81033b0 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8f9f400640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8face00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 0x7f8fa81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:03.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa8199490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f98036040 tx=0x7f8f98004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa5600640 1 -- 172.21.15.6:0/2445354572 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98004aa0 con 0x7f8fa81033b0 2024-02-26T06:27:03.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa5600640 1 -- 172.21.15.6:0/2445354572 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f980052d0 con 0x7f8fa81033b0 2024-02-26T06:27:03.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa81a1530 con 0x7f8fa81033b0 2024-02-26T06:27:03.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8fa5600640 1 -- 172.21.15.6:0/2445354572 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98003dc0 con 0x7f8fa81033b0 2024-02-26T06:27:03.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.478+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa81a1a10 con 0x7f8fa81033b0 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.481+0000 7f8fb1da5740 1 Processor -- start 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fa5600640 1 -- 172.21.15.6:0/2445354572 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8f98005440 con 0x7f8fa81033b0 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 -- start start 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba12970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57b981b00 con 0x55c57ba12550 2024-02-26T06:27:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57b97f030 con 0x55c57ba12f40 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57b97a810 con 0x55c57ba180b0 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8face00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba12970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8face00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba12970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38450/0 (socket says 172.21.15.6:38450) 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8face00640 1 -- 172.21.15.6:0/1463903028 learned_addr learned my addr 172.21.15.6:0/1463903028 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.482+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8face00640 1 -- 172.21.15.6:0/1463903028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 msgr2=0x55c57ba1a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8face00640 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8face00640 1 -- 172.21.15.6:0/1463903028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 msgr2=0x55c57ba17a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8face00640 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba17a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8face00640 1 -- 172.21.15.6:0/1463903028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c57b942490 con 0x55c57ba12550 2024-02-26T06:27:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba17a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8face00640 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba12970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00107e0 tx=0x7f8fa0010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bc6d97b28eee90 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8f9d600640 1 -- 172.21.15.6:0/1463903028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0003b70 con 0x55c57ba12550 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8f9d600640 1 -- 172.21.15.6:0/1463903028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0015070 con 0x55c57ba12550 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8f9d600640 1 -- 172.21.15.6:0/1463903028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0004630 con 0x55c57ba12550 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.483+0000 7f8fa5600640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8f8c0a20c0 0x7f8f8c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fa5600640 1 -- 172.21.15.6:0/2445354572 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8f980f0050 con 0x7f8fa81033b0 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8f8c0a20c0 0x7f8f8c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1463903028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 msgr2=0x55c57ba12970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba12970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00107e0 tx=0x7f8fa0010ca0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1463903028 shutdown_connections 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba17a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1463903028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba12970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1463903028 >> 172.21.15.6:0/1463903028 conn(0x55c57ba0dac0 msgr2=0x55c57ba0dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1463903028 shutdown_connections 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1463903028 wait complete. 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.484+0000 7f8fa7e00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8f8c0a20c0 0x7f8f8c0a4580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f980edeb0 tx=0x7f8f980f3af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 -- start start 2024-02-26T06:27:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 Processor -- start 2024-02-26T06:27:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba1c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12f40 0x55c57ba1c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57b91fe70 con 0x55c57ba12f40 2024-02-26T06:27:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57b91fb10 con 0x55c57ba12550 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57b6c2f20 con 0x55c57ba180b0 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.485+0000 7f8fb1da5740 10 client.0 _ll_register_callbacks cb 0x55c57b933920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8face00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba1c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12f40 0x55c57ba1c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8face00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba1c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33244/0 (socket says 172.21.15.6:33244) 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8face00640 1 -- 172.21.15.6:0/1006868132 learned_addr learned my addr 172.21.15.6:0/1006868132 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8face00640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 -- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 msgr2=0x55c57ba238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba238f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 -- 172.21.15.6:0/1006868132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 msgr2=0x55c57ba1c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba1c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 -- 172.21.15.6:0/1006868132 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa0010410 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8face00640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba1c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.486+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12f40 0x55c57ba1c8b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009640 tx=0x7f8f9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b63e19927513108 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8f96000640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8f96000640 1 -- 172.21.15.6:0/1006868132 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90017070 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8f96000640 1 -- 172.21.15.6:0/1006868132 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8f900120d0 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c57b99ec10 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8f96000640 1 -- 172.21.15.6:0/1006868132 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f90012290 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8fb1da5740 10 client.14868 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c57b978e90 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.487+0000 7f8f96000640 1 -- 172.21.15.6:0/1006868132 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c57b99ec10 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8f96000640 1 -- 172.21.15.6:0/1006868132 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f90010660 con 0x55c57ba12f40 2024-02-26T06:27:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8fb1da5740 10 client.14868 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8fb1da5740 10 client.14868 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8fb1da5740 20 client.14868 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba283c0 0x55c57ba27c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8fb1da5740 4 client.14868 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c57b67a6c0 con 0x55c57ba283c0 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.488+0000 7f8face00640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba283c0 0x55c57ba27c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.489+0000 7f8face00640 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba283c0 0x55c57ba27c90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=422343bfa9ddde4b server_cookie=6d6cf9c6e552c103 in_seq=0 out_seq=0 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.489+0000 7f8f96000640 10 client.14868 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.489+0000 7f8f96000640 1 -- 172.21.15.6:0/1006868132 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c57b67a6c0 con 0x55c57ba283c0 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.489+0000 7f8f96000640 10 client.14868 handle_command_reply: tid=0 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.489+0000 7f8fb1da5740 1 client.14868 shutdown 2024-02-26T06:27:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12f40 msgr2=0x55c57ba1c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12f40 0x55c57ba1c8b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009640 tx=0x7f8f9000f790 comp rx=0 tx=0).stop 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 shutdown_connections 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba283c0 0x55c57ba27c90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba238f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12f40 0x55c57ba1c8b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8f96000640 0 client.14868 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1006868132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba1c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 >> 172.21.15.6:0/1006868132 conn(0x55c57ba0dac0 msgr2=0x55c57ba0cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 shutdown_connections 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1006868132 wait complete. 2024-02-26T06:27:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.490+0000 7f8fb1da5740 20 client.14868 trim_cache size 0 max 16384 2024-02-26T06:27:03.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:03 smithi167 ceph-mon[36229]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:03.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:03 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/336842230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:03.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:03 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3029290891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:03.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 Processor -- start 2024-02-26T06:27:03.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 -- start start 2024-02-26T06:27:03.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba0dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57ba28c30 con 0x55c57ba12550 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57ba241b0 con 0x55c57ba12f40 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.507+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57ba13630 con 0x55c57ba180b0 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8face00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba0dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33248/0 (socket says 172.21.15.6:33248) 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 -- 172.21.15.6:0/1142817623 learned_addr learned my addr 172.21.15.6:0/1142817623 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 -- 172.21.15.6:0/1142817623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 msgr2=0x55c57ba1bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 -- 172.21.15.6:0/1142817623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 msgr2=0x55c57ba0dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba0dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 -- 172.21.15.6:0/1142817623 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c57b915880 con 0x55c57ba12f40 2024-02-26T06:27:03.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba238b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f90010b10 tx=0x7f8f900059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383f0072ff71b73c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.508+0000 7f8f9cc00640 1 -- 172.21.15.6:0/1142817623 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90017040 con 0x55c57ba12f40 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8f9cc00640 1 -- 172.21.15.6:0/1142817623 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f90005040 con 0x55c57ba12f40 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8f9cc00640 1 -- 172.21.15.6:0/1142817623 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90004a40 con 0x55c57ba12f40 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1142817623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 msgr2=0x55c57ba238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba238b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f90010b10 tx=0x7f8f900059b0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1142817623 shutdown_connections 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba180b0 0x55c57ba1bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12f40 0x55c57ba238b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1142817623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba12550 0x55c57ba0dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1142817623 >> 172.21.15.6:0/1142817623 conn(0x55c57ba0dac0 msgr2=0x55c57ba08c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1142817623 shutdown_connections 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.509+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1142817623 wait complete. 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 -- start start 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 Processor -- start 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 0x55c57ba20360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba180b0 0x55c57ba208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57ba20fc0 con 0x55c57ba180b0 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57ba7e630 con 0x55c57ba12550 2024-02-26T06:27:03.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57ba7e7f0 con 0x55c57ba12f40 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8fb1da5740 10 client.0 _ll_register_callbacks cb 0x7f8f90012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8face00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.510+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 0x55c57ba20360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8face00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba180b0 0x55c57ba208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 0x55c57ba20360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49156/0 (socket says 172.21.15.6:49156) 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 -- 172.21.15.6:0/1510636821 learned_addr learned my addr 172.21.15.6:0/1510636821 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 -- 172.21.15.6:0/1510636821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 msgr2=0x55c57ba238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba238c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 -- 172.21.15.6:0/1510636821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba180b0 msgr2=0x55c57ba208c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba180b0 0x55c57ba208c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 -- 172.21.15.6:0/1510636821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c57b978e90 con 0x55c57ba12f40 2024-02-26T06:27:03.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f9f400640 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 0x55c57ba20360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f9002d000 tx=0x7f8f9000ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a3edcbee148708 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f96000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f96000640 1 -- 172.21.15.6:0/1510636821 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90017040 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8f96000640 1 -- 172.21.15.6:0/1510636821 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8f90004780 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.511+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c57ba28d70 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8f96000640 1 -- 172.21.15.6:0/1510636821 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f90010760 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 10 client.25000 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c57b93b4e0 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8f96000640 1 -- 172.21.15.6:0/1510636821 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f90041080 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8f96000640 1 -- 172.21.15.6:0/1510636821 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f9001cb00 con 0x55c57ba12f40 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 10 client.25000 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 10 client.25000 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 20 client.25000 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:03.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba83940 0x55c57ba7fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 4 client.25000 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.512+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f8fa0015070 con 0x55c57ba83940 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.513+0000 7f8face00640 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba83940 0x55c57ba7fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.513+0000 7f8face00640 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba83940 0x55c57ba7fdc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0ab397eefee023e server_cookie=eaec5eaedccaaeb2 in_seq=0 out_seq=0 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.513+0000 7f8f96000640 10 client.25000 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.513+0000 7f8f96000640 1 -- 172.21.15.6:0/1510636821 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f8fa0015070 con 0x55c57ba83940 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.513+0000 7f8f96000640 10 client.25000 handle_command_reply: tid=0 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.513+0000 7f8fb1da5740 1 client.25000 shutdown 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 msgr2=0x55c57ba20360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 0x55c57ba20360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f9002d000 tx=0x7f8f9000ffa0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 shutdown_connections 2024-02-26T06:27:03.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55c57ba83940 0x55c57ba7fdc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ba180b0 0x55c57ba208c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57ba12f40 0x55c57ba20360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8f96000640 0 client.25000 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 --2- 172.21.15.6:0/1510636821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57ba12550 0x55c57ba238c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 >> 172.21.15.6:0/1510636821 conn(0x55c57ba0dac0 msgr2=0x55c57ba0dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 shutdown_connections 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 1 -- 172.21.15.6:0/1510636821 wait complete. 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.514+0000 7f8fb1da5740 20 client.25000 trim_cache size 0 max 16384 2024-02-26T06:27:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 4, 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24520, 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 99.176532657999999, 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:03.521 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 129.21482270800001 2024-02-26T06:27:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:03.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8f8c0a20c0 msgr2=0x7f8f8c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8f8c0a20c0 0x7f8f8c0a4580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f980edeb0 tx=0x7f8f980f3af0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 msgr2=0x7f8fa8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa8199490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f98036040 tx=0x7f8f98004270 comp rx=0 tx=0).stop 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 shutdown_connections 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fa8104f60 0x7f8fa81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f8f8c0a20c0 0x7f8f8c0a4580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa81045a0 0x7f8fa81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 --2- 172.21.15.6:0/2445354572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fa81033b0 0x7f8fa8199490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 >> 172.21.15.6:0/2445354572 conn(0x7f8fa80fe940 msgr2=0x7f8fa8107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 shutdown_connections 2024-02-26T06:27:03.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.515+0000 7f8faee00640 1 -- 172.21.15.6:0/2445354572 wait complete. 2024-02-26T06:27:03.534 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-26T06:27:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 -- 172.21.15.6:0/638198660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b01021b0 msgr2=0x7f20b0102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 --2- 172.21.15.6:0/638198660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b01021b0 0x7f20b0102630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f209c009900 tx=0x7f209c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 -- 172.21.15.6:0/638198660 shutdown_connections 2024-02-26T06:27:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 --2- 172.21.15.6:0/638198660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b0102b70 0x7f20b0109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 --2- 172.21.15.6:0/638198660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b01021b0 0x7f20b0102630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 --2- 172.21.15.6:0/638198660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b0100fc0 0x7f20b01013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.762+0000 7f20af200640 1 -- 172.21.15.6:0/638198660 >> 172.21.15.6:0/638198660 conn(0x7f20b00fc530 msgr2=0x7f20b00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.763+0000 7f20af200640 1 -- 172.21.15.6:0/638198660 shutdown_connections 2024-02-26T06:27:03.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.763+0000 7f20af200640 1 -- 172.21.15.6:0/638198660 wait complete. 2024-02-26T06:27:03.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 Processor -- start 2024-02-26T06:27:03.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 -- start start 2024-02-26T06:27:03.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b0100fc0 0x7f20b0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b01021b0 0x7f20b01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 0x7f20b01a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01a0d70 con 0x7f20b0102b70 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01a0ee0 con 0x7f20b01021b0 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20af200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01a10a0 con 0x7f20b0100fc0 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20ae800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 0x7f20b01a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20ae800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 0x7f20b01a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38496/0 (socket says 172.21.15.6:38496) 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20ae800640 1 -- 172.21.15.6:0/3296954242 learned_addr learned my addr 172.21.15.6:0/3296954242 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20ad400640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b01021b0 0x7f20b01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.764+0000 7f20ade00640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b0100fc0 0x7f20b0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20ae800640 1 -- 172.21.15.6:0/3296954242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b0100fc0 msgr2=0x7f20b0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20ae800640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b0100fc0 0x7f20b0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20ae800640 1 -- 172.21.15.6:0/3296954242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b01021b0 msgr2=0x7f20b01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20ae800640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b01021b0 0x7f20b01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20ae800640 1 -- 172.21.15.6:0/3296954242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009490 con 0x7f20b0102b70 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20ae800640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 0x7f20b01a0830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f20a800eb10 tx=0x7f20a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20a6a00640 1 -- 172.21.15.6:0/3296954242 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8004b20 con 0x7f20b0102b70 2024-02-26T06:27:03.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20a6a00640 1 -- 172.21.15.6:0/3296954242 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20a8004c80 con 0x7f20b0102b70 2024-02-26T06:27:03.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20a6a00640 1 -- 172.21.15.6:0/3296954242 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8010650 con 0x7f20b0102b70 2024-02-26T06:27:03.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.765+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c009560 con 0x7f20b0102b70 2024-02-26T06:27:03.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.766+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b01a1680 con 0x7f20b0102b70 2024-02-26T06:27:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 Processor -- start 2024-02-26T06:27:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20a6a00640 1 -- 172.21.15.6:0/3296954242 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f20a8005020 con 0x7f20b0102b70 2024-02-26T06:27:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 -- start start 2024-02-26T06:27:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a2cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a31c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a34720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9399bb00 con 0x562f93a2c730 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93999030 con 0x562f93a2d120 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.769+0000 7f20b79b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93994810 con 0x562f93a32290 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a31c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a31c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33284/0 (socket says 172.21.15.6:33284) 2024-02-26T06:27:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 -- 172.21.15.6:0/2300743221 learned_addr learned my addr 172.21.15.6:0/2300743221 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ad400640 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a2cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 -- 172.21.15.6:0/2300743221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 msgr2=0x562f93a34720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a34720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 -- 172.21.15.6:0/2300743221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 msgr2=0x562f93a2cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a2cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 -- 172.21.15.6:0/2300743221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f9395c490 con 0x562f93a2d120 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f20ade00640 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a31c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009e20 tx=0x7f20a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b174b2ad832d31de server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.770+0000 7f209b000640 1 -- 172.21.15.6:0/2300743221 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0019630 con 0x562f93a2d120 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f209b000640 1 -- 172.21.15.6:0/2300743221 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20a0019c70 con 0x562f93a2d120 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f209b000640 1 -- 172.21.15.6:0/2300743221 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a000ecf0 con 0x562f93a2d120 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20a6a00640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20940a2140 0x7f20940a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20a6a00640 1 -- 172.21.15.6:0/3296954242 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f20a8014070 con 0x7f20b0102b70 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20ad400640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20940a2140 0x7f20940a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 -- 172.21.15.6:0/2300743221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 msgr2=0x562f93a31c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a31c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009e20 tx=0x7f20a0011810 comp rx=0 tx=0).stop 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 -- 172.21.15.6:0/2300743221 shutdown_connections 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a34720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a31c20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2300743221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a2cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.771+0000 7f20b79b3740 1 -- 172.21.15.6:0/2300743221 >> 172.21.15.6:0/2300743221 conn(0x562f93a27ca0 msgr2=0x562f93a280a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.772+0000 7f20b79b3740 1 -- 172.21.15.6:0/2300743221 shutdown_connections 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.772+0000 7f20ad400640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20940a2140 0x7f20940a4600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f209c009230 tx=0x7f209c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.772+0000 7f20b79b3740 1 -- 172.21.15.6:0/2300743221 wait complete. 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.772+0000 7f20b79b3740 1 -- start start 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.772+0000 7f20b79b3740 1 Processor -- start 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a369b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a3d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93939e70 con 0x562f93a2c730 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93939b10 con 0x562f93a2d120 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f936dcf20 con 0x562f93a32290 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20b79b3740 10 client.0 _ll_register_callbacks cb 0x562f9394d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ade00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ad400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ad400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a369b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33298/0 (socket says 172.21.15.6:33298) 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ad400640 1 -- 172.21.15.6:0/602439801 learned_addr learned my addr 172.21.15.6:0/602439801 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ae800640 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a3d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ade00640 1 -- 172.21.15.6:0/602439801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 msgr2=0x562f93a3d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ade00640 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a3d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ade00640 1 -- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 msgr2=0x562f93a369b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ade00640 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a369b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.773+0000 7f20ade00640 1 -- 172.21.15.6:0/602439801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a0009a50 con 0x562f93a2c730 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f20ade00640 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a36470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x562f93a28c10 tx=0x7f20a000eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ac98b3760a94c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f208f400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f208f400640 1 -- 172.21.15.6:0/602439801 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0004220 con 0x562f93a2c730 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f939b8c10 con 0x562f93a2c730 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f208f400640 1 -- 172.21.15.6:0/602439801 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f20a0004380 con 0x562f93a2c730 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f208f400640 1 -- 172.21.15.6:0/602439801 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a000eeb0 con 0x562f93a2c730 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f20b79b3740 10 client.14886 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562f93992e90 con 0x562f93a2c730 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.774+0000 7f208f400640 1 -- 172.21.15.6:0/602439801 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562f939b8c10 con 0x562f93a2c730 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f208f400640 1 -- 172.21.15.6:0/602439801 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f20a00046c0 con 0x562f93a2c730 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20b79b3740 10 client.14886 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20b79b3740 10 client.14886 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20b79b3740 20 client.14886 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20b79b3740 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a428d0 0x562f93a46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20b79b3740 4 client.14886 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f936946c0 con 0x562f93a428d0 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.775+0000 7f20ad400640 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a428d0 0x562f93a46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.776+0000 7f20ad400640 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a428d0 0x562f93a46d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26681dd9e15f9810 server_cookie=3e87305740d1833b in_seq=0 out_seq=0 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.776+0000 7f208f400640 10 client.14886 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f208f400640 1 -- 172.21.15.6:0/602439801 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f936946c0 con 0x562f93a428d0 2024-02-26T06:27:03.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f208f400640 10 client.14886 handle_command_reply: tid=0 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 client.14886 shutdown 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 msgr2=0x562f93a36470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a36470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x562f93a28c10 tx=0x7f20a000eab0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 shutdown_connections 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a428d0 0x562f93a46d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a32290 0x562f93a3d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93a369b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 --2- 172.21.15.6:0/602439801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a36470 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f208f400640 0 client.14886 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:03.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 >> 172.21.15.6:0/602439801 conn(0x562f93a27ca0 msgr2=0x562f93a26de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 shutdown_connections 2024-02-26T06:27:03.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 1 -- 172.21.15.6:0/602439801 wait complete. 2024-02-26T06:27:03.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.777+0000 7f20b79b3740 20 client.14886 trim_cache size 0 max 16384 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.794+0000 7f20b79b3740 1 Processor -- start 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.794+0000 7f20b79b3740 1 -- start start 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.794+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a27ca0 0x562f93a4ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a2d120 0x562f93a4b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20b79b3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93a43160 con 0x562f93a2c730 2024-02-26T06:27:03.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20b79b3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93a437c0 con 0x562f93a27ca0 2024-02-26T06:27:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20b79b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93a3e2b0 con 0x562f93a2d120 2024-02-26T06:27:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38512/0 (socket says 172.21.15.6:38512) 2024-02-26T06:27:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 -- 172.21.15.6:0/2538564865 learned_addr learned my addr 172.21.15.6:0/2538564865 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ae800640 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a2d120 0x562f93a4b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ade00640 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a27ca0 0x562f93a4ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 -- 172.21.15.6:0/2538564865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a2d120 msgr2=0x562f93a4b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a2d120 0x562f93a4b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 -- 172.21.15.6:0/2538564865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a27ca0 msgr2=0x562f93a4ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a27ca0 0x562f93a4ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.795+0000 7f20ad400640 1 -- 172.21.15.6:0/2538564865 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f9392f880 con 0x562f93a2c730 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.796+0000 7f20ad400640 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3d980 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f209c005ab0 tx=0x7f209c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b38c20bc9fc57 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.796+0000 7f209a600640 1 -- 172.21.15.6:0/2538564865 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c004ae0 con 0x562f93a2c730 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.796+0000 7f209a600640 1 -- 172.21.15.6:0/2538564865 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f209c053e90 con 0x562f93a2c730 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.796+0000 7f209a600640 1 -- 172.21.15.6:0/2538564865 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c003d30 con 0x562f93a2c730 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 -- 172.21.15.6:0/2538564865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 msgr2=0x562f93a3d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3d980 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f209c005ab0 tx=0x7f209c045000 comp rx=0 tx=0).stop 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 -- 172.21.15.6:0/2538564865 shutdown_connections 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a2d120 0x562f93a4b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3d980 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 --2- 172.21.15.6:0/2538564865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a27ca0 0x562f93a4ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 -- 172.21.15.6:0/2538564865 >> 172.21.15.6:0/2538564865 conn(0x562f93a283f0 msgr2=0x562f93a26de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 -- 172.21.15.6:0/2538564865 shutdown_connections 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 -- 172.21.15.6:0/2538564865 wait complete. 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 -- start start 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.797+0000 7f20b79b3740 1 Processor -- start 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 0x562f93a3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93aa1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93a3b1a0 con 0x562f93a2c730 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93a3b310 con 0x562f93a2d120 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f93aa1760 con 0x562f93a27ca0 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20b79b3740 10 client.0 _ll_register_callbacks cb 0x7f20a0009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20ad400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20ade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 0x562f93a3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20ad400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38526/0 (socket says 172.21.15.6:38526) 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20ae800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93aa1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20ad400640 1 -- 172.21.15.6:0/3048691233 learned_addr learned my addr 172.21.15.6:0/3048691233 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.798+0000 7f20ade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 0x562f93a3a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49196/0 (socket says 172.21.15.6:49196) 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ad400640 1 -- 172.21.15.6:0/3048691233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 msgr2=0x562f93a3a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ad400640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 0x562f93a3a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ad400640 1 -- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 msgr2=0x562f93aa1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ad400640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93aa1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ad400640 1 -- 172.21.15.6:0/3048691233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0109d00 con 0x562f93a2c730 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ae800640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93aa1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ade00640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 0x562f93a3a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20ad400640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3ac60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f209c004830 tx=0x7f209c02ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf631251a51272c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f208f400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f208f400640 1 -- 172.21.15.6:0/3048691233 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c003cb0 con 0x562f93a2c730 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f93a43900 con 0x562f93a2c730 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.799+0000 7f208f400640 1 -- 172.21.15.6:0/3048691233 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f209c049040 con 0x562f93a2c730 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f208f400640 1 -- 172.21.15.6:0/3048691233 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f209c003e10 con 0x562f93a2c730 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 10 client.14892 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562f939554e0 con 0x562f93a2c730 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f208f400640 1 -- 172.21.15.6:0/3048691233 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f209c06d020 con 0x562f93a2c730 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f208f400640 1 -- 172.21.15.6:0/3048691233 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f209c03c040 con 0x562f93a2c730 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 10 client.14892 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 10 client.14892 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 20 client.14892 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a32290 0x562f93aa2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 4 client.14892 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.800+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f93a432a0 con 0x562f93a32290 2024-02-26T06:27:03.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20ade00640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a32290 0x562f93aa2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:03.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20ade00640 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a32290 0x562f93aa2bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe23215af03d0c1f server_cookie=c8766b7f0b406dd3 in_seq=0 out_seq=0 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f208f400640 10 client.14892 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f208f400640 1 -- 172.21.15.6:0/3048691233 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562f93a432a0 con 0x562f93a32290 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f208f400640 10 client.14892 handle_command_reply: tid=0 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 client.14892 shutdown 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 msgr2=0x562f93a3ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3ac60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f209c004830 tx=0x7f209c02ccd0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 shutdown_connections 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x562f93a32290 0x562f93aa2bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f93a2d120 0x562f93aa1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f93a2c730 0x562f93a3ac60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f20b79b3740 1 --2- 172.21.15.6:0/3048691233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f93a27ca0 0x562f93a3a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.801+0000 7f208f400640 0 client.14892 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 >> 172.21.15.6:0/3048691233 conn(0x562f93a283f0 msgr2=0x562f93a4ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 shutdown_connections 2024-02-26T06:27:03.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20b79b3740 1 -- 172.21.15.6:0/3048691233 wait complete. 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20b79b3740 20 client.14892 trim_cache size 0 max 16384 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 3, 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24530, 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 100.50410390499999, 2024-02-26T06:27:03.808 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 126.782152814 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20940a2140 msgr2=0x7f20940a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20940a2140 0x7f20940a4600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f209c009230 tx=0x7f209c005bd0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 msgr2=0x7f20b01a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 0x7f20b01a0830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f20a800eb10 tx=0x7f20a800c2d0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 shutdown_connections 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20b0102b70 0x7f20b01a0830 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20940a2140 0x7f20940a4600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b01021b0 0x7f20b01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 --2- 172.21.15.6:0/3296954242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b0100fc0 0x7f20b0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.802+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 >> 172.21.15.6:0/3296954242 conn(0x7f20b00fc530 msgr2=0x7f20b00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.803+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 shutdown_connections 2024-02-26T06:27:03.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:03.803+0000 7f20af200640 1 -- 172.21.15.6:0/3296954242 wait complete. 2024-02-26T06:27:03.821 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 -- 172.21.15.6:0/1566156391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 msgr2=0x7f4c0c073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 --2- 172.21.15.6:0/1566156391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c073610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009900 tx=0x7f4bfc02bc50 comp rx=0 tx=0).stop 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 -- 172.21.15.6:0/1566156391 shutdown_connections 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 --2- 172.21.15.6:0/1566156391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c0c108e80 0x7f4c0c10b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 --2- 172.21.15.6:0/1566156391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c073be0 0x7f4c0c074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 --2- 172.21.15.6:0/1566156391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c073610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 -- 172.21.15.6:0/1566156391 >> 172.21.15.6:0/1566156391 conn(0x7f4c0c0fc530 msgr2=0x7f4c0c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 -- 172.21.15.6:0/1566156391 shutdown_connections 2024-02-26T06:27:04.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.035+0000 7f4c0be00640 1 -- 172.21.15.6:0/1566156391 wait complete. 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 Processor -- start 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 -- start start 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c073be0 0x7f4c0c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c0c108e80 0x7f4c0c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1a0dc0 con 0x7f4c0c0751c0 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1a0f30 con 0x7f4c0c108e80 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1a10f0 con 0x7f4c0c073be0 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38544/0 (socket says 172.21.15.6:38544) 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c03e00640 1 -- 172.21.15.6:0/4022907364 learned_addr learned my addr 172.21.15.6:0/4022907364 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.037+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c073be0 0x7f4c0c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c0b400640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c0c108e80 0x7f4c0c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03e00640 1 -- 172.21.15.6:0/4022907364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c073be0 msgr2=0x7f4c0c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03e00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c073be0 0x7f4c0c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03e00640 1 -- 172.21.15.6:0/4022907364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c0c108e80 msgr2=0x7f4c0c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03e00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c0c108e80 0x7f4c0c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03e00640 1 -- 172.21.15.6:0/4022907364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04009840 con 0x7f4c0c0751c0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03e00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c199810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e9f0 tx=0x7f4bf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03400640 1 -- 172.21.15.6:0/4022907364 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4009720 con 0x7f4c0c0751c0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03400640 1 -- 172.21.15.6:0/4022907364 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bf4005500 con 0x7f4c0c0751c0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c03400640 1 -- 172.21.15.6:0/4022907364 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4010430 con 0x7f4c0c0751c0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.038+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc009560 con 0x7f4c0c0751c0 2024-02-26T06:27:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.039+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c1a1750 con 0x7f4c0c0751c0 2024-02-26T06:27:04.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 Processor -- start 2024-02-26T06:27:04.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c03400640 1 -- 172.21.15.6:0/4022907364 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4bf4005020 con 0x7f4c0c0751c0 2024-02-26T06:27:04.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 -- start start 2024-02-26T06:27:04.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa91045b00 con 0x55fa910d6730 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa91043030 con 0x55fa910dc290 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.042+0000 7f4c1476c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9103e810 con 0x55fa910d7120 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910d6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38550/0 (socket says 172.21.15.6:38550) 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 -- 172.21.15.6:0/1073449100 learned_addr learned my addr 172.21.15.6:0/1073449100 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c03e00640 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 -- 172.21.15.6:0/1073449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 msgr2=0x55fa910dbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910dbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 -- 172.21.15.6:0/1073449100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 msgr2=0x55fa910de720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910de720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0b400640 1 -- 172.21.15.6:0/1073449100 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa91006490 con 0x55fa910d6730 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.043+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4c0b400640 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910d6b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c040115b0 tx=0x7f4c04011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c7ce7182697b0a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4c03400640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4bf00a2140 0x7f4bf00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4bfbe00640 1 -- 172.21.15.6:0/1073449100 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04004d20 con 0x55fa910d6730 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4bfbe00640 1 -- 172.21.15.6:0/1073449100 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c04004e80 con 0x55fa910d6730 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4bfbe00640 1 -- 172.21.15.6:0/1073449100 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400f860 con 0x55fa910d6730 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4c03400640 1 -- 172.21.15.6:0/4022907364 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4bf4014070 con 0x7f4c0c0751c0 2024-02-26T06:27:04.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.044+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4bf00a2140 0x7f4bf00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 -- 172.21.15.6:0/1073449100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 msgr2=0x55fa910d6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910d6b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c040115b0 tx=0x7f4c04011a70 comp rx=0 tx=0).stop 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 -- 172.21.15.6:0/1073449100 shutdown_connections 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1073449100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910d6b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 -- 172.21.15.6:0/1073449100 >> 172.21.15.6:0/1073449100 conn(0x55fa910d1ca0 msgr2=0x55fa910d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 -- 172.21.15.6:0/1073449100 shutdown_connections 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4bf00a2140 0x7f4bf00a4600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009230 tx=0x7f4bfc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 -- 172.21.15.6:0/1073449100 wait complete. 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.045+0000 7f4c1476c740 1 -- start start 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 Processor -- start 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa90fe3e70 con 0x55fa910d6730 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa90fe3b10 con 0x55fa910dc290 2024-02-26T06:27:04.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa90d86f20 con 0x55fa910d7120 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c1476c740 10 client.0 _ll_register_callbacks cb 0x55fa90ff7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.046+0000 7f4c0aa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910e0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38558/0 (socket says 172.21.15.6:38558) 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 -- 172.21.15.6:0/1142110212 learned_addr learned my addr 172.21.15.6:0/1142110212 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c03e00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0b400640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 -- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 msgr2=0x55fa910e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 -- 172.21.15.6:0/1142110212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 msgr2=0x55fa910e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 -- 172.21.15.6:0/1142110212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c04009e20 con 0x55fa910d6730 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c03e00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910e0530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc035000 tx=0x7f4bfc02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa13c217c60adcf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.047+0000 7f4bf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:04.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4bf8c00640 1 -- 172.21.15.6:0/1142110212 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0043d0 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4bf8c00640 1 -- 172.21.15.6:0/1142110212 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4bfc004df0 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa90d3e6c0 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4bf8c00640 1 -- 172.21.15.6:0/1142110212 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bfc004fb0 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4c1476c740 10 client.14904 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fa9103ce90 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4bf8c00640 1 -- 172.21.15.6:0/1142110212 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fa90d3e6c0 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.048+0000 7f4bf8c00640 1 -- 172.21.15.6:0/1142110212 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bfc03c040 con 0x55fa910d6730 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c1476c740 10 client.14904 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c1476c740 10 client.14904 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c1476c740 20 client.14904 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa910ebdf0 0x55fa910f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c1476c740 4 client.14904 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa90d8f270 con 0x55fa910ebdf0 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.049+0000 7f4c03e00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa910ebdf0 0x55fa910f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.050+0000 7f4c03e00640 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa910ebdf0 0x55fa910f0e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ae97c71a22e6757 server_cookie=baa3d5bb49bd8756 in_seq=0 out_seq=0 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.050+0000 7f4bf8c00640 10 client.14904 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4bf8c00640 1 -- 172.21.15.6:0/1142110212 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa90d8f270 con 0x55fa910ebdf0 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4bf8c00640 10 client.14904 handle_command_reply: tid=0 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 client.14904 shutdown 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 msgr2=0x55fa910e0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910e0530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc035000 tx=0x7f4bfc02bc50 comp rx=0 tx=0).stop 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 shutdown_connections 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa910ebdf0 0x55fa910f0e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d7120 0x55fa910e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 --2- 172.21.15.6:0/1142110212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d6730 0x55fa910e0530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4bf8c00640 0 client.14904 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 >> 172.21.15.6:0/1142110212 conn(0x55fa910d1ca0 msgr2=0x55fa910d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 shutdown_connections 2024-02-26T06:27:04.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 1 -- 172.21.15.6:0/1142110212 wait complete. 2024-02-26T06:27:04.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.051+0000 7f4c1476c740 20 client.14904 trim_cache size 0 max 16384 2024-02-26T06:27:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 Processor -- start 2024-02-26T06:27:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 -- start start 2024-02-26T06:27:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 0x55fa910e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910dc290 0x55fa910f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa910e8aa0 con 0x55fa910dc290 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa910df690 con 0x55fa910d7120 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.069+0000 7f4c1476c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa910ea9a0 con 0x55fa910d6730 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 0x55fa910e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 0x55fa910e7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33374/0 (socket says 172.21.15.6:33374) 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 -- 172.21.15.6:0/2548774846 learned_addr learned my addr 172.21.15.6:0/2548774846 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c0b400640 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910dc290 0x55fa910f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 -- 172.21.15.6:0/2548774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 msgr2=0x55fa910d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 -- 172.21.15.6:0/2548774846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910dc290 msgr2=0x55fa910f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910dc290 0x55fa910f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 -- 172.21.15.6:0/2548774846 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa90fd9880 con 0x55fa910d7120 2024-02-26T06:27:04.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.070+0000 7f4c03e00640 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 0x55fa910e7a40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4097690 tx=0x7f4bf40cb9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8d60fd17824165 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.074+0000 7f4bfb400640 1 -- 172.21.15.6:0/2548774846 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf409a850 con 0x55fa910d7120 2024-02-26T06:27:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.074+0000 7f4bfb400640 1 -- 172.21.15.6:0/2548774846 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bf409a670 con 0x55fa910d7120 2024-02-26T06:27:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.074+0000 7f4bfb400640 1 -- 172.21.15.6:0/2548774846 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40a34b0 con 0x55fa910d7120 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 -- 172.21.15.6:0/2548774846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 msgr2=0x55fa910e7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 0x55fa910e7a40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4097690 tx=0x7f4bf40cb9b0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 -- 172.21.15.6:0/2548774846 shutdown_connections 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910dc290 0x55fa910f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910d7120 0x55fa910e7a40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 --2- 172.21.15.6:0/2548774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.075+0000 7f4c1476c740 1 -- 172.21.15.6:0/2548774846 >> 172.21.15.6:0/2548774846 conn(0x55fa910d1ca0 msgr2=0x55fa910d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.076+0000 7f4c1476c740 1 -- 172.21.15.6:0/2548774846 shutdown_connections 2024-02-26T06:27:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.076+0000 7f4c1476c740 1 -- 172.21.15.6:0/2548774846 wait complete. 2024-02-26T06:27:04.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.076+0000 7f4c1476c740 1 -- start start 2024-02-26T06:27:04.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.076+0000 7f4c1476c740 1 Processor -- start 2024-02-26T06:27:04.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910e7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 0x55fa910e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa910e4630 con 0x55fa910d7120 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa910e47a0 con 0x55fa910dc290 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa910e4960 con 0x55fa910d6730 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c1476c740 10 client.0 _ll_register_callbacks cb 0x7f4bfc043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 0x55fa910e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c0aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 0x55fa910e79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38568/0 (socket says 172.21.15.6:38568) 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 -- 172.21.15.6:0/3012420595 learned_addr learned my addr 172.21.15.6:0/3012420595 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c0aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910e7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49246/0 (socket says 172.21.15.6:49246) 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c0b400640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 -- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 msgr2=0x55fa910e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 -- 172.21.15.6:0/3012420595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 msgr2=0x55fa910e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c03e00640 1 -- 172.21.15.6:0/3012420595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c1a1750 con 0x55fa910d7120 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.077+0000 7f4c0b400640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4c03e00640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 0x55fa910e79d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40cbe50 tx=0x7f4bf409a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d384f6a9aeab72f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4bf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4bf8c00640 1 -- 172.21.15.6:0/3012420595 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4005670 con 0x55fa910d7120 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4bf8c00640 1 -- 172.21.15.6:0/3012420595 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4bf409b410 con 0x55fa910d7120 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa910eda20 con 0x55fa910d7120 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4bf8c00640 1 -- 172.21.15.6:0/3012420595 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bf40a34b0 con 0x55fa910d7120 2024-02-26T06:27:04.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4c1476c740 10 client.14910 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.078+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fa90fff4e0 con 0x55fa910d7120 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4bf8c00640 1 -- 172.21.15.6:0/3012420595 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bf40a7020 con 0x55fa910d7120 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4bf8c00640 1 -- 172.21.15.6:0/3012420595 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bf401f060 con 0x55fa910d7120 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c1476c740 10 client.14910 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c1476c740 10 client.14910 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c1476c740 20 client.14910 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c1476c740 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa91147880 0x55fa91143d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c1476c740 4 client.14910 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fa91062c10 con 0x55fa91147880 2024-02-26T06:27:04.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.079+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa91147880 0x55fa91143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c0aa00640 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa91147880 0x55fa91143d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=669d7e8fc14bb57b server_cookie=71680be5062b5bc4 in_seq=0 out_seq=0 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4bf8c00640 10 client.14910 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4bf8c00640 1 -- 172.21.15.6:0/3012420595 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fa91062c10 con 0x55fa91147880 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4bf8c00640 10 client.14910 handle_command_reply: tid=0 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 client.14910 shutdown 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 msgr2=0x55fa910e79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 0x55fa910e79d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40cbe50 tx=0x7f4bf409a8d0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 shutdown_connections 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55fa91147880 0x55fa91143d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa910dc290 0x55fa910e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55fa910d7120 0x55fa910e79d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 --2- 172.21.15.6:0/3012420595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa910d6730 0x55fa910e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4bf8c00640 0 client.14910 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.080+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 >> 172.21.15.6:0/3012420595 conn(0x55fa910d1ca0 msgr2=0x55fa910d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 shutdown_connections 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c1476c740 1 -- 172.21.15.6:0/3012420595 wait complete. 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c1476c740 20 client.14910 trim_cache size 0 max 16384 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-02-26T06:27:04.086 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24544, 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 101.79500770200001, 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 124.234209695 2024-02-26T06:27:04.087 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4bf00a2140 msgr2=0x7f4bf00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4bf00a2140 0x7f4bf00a4600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009230 tx=0x7f4bfc005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 msgr2=0x7f4c0c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c199810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e9f0 tx=0x7f4bf400edf0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 shutdown_connections 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4bf00a2140 0x7f4bf00a4600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c0c108e80 0x7f4c0c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c0751c0 0x7f4c0c199810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 --2- 172.21.15.6:0/4022907364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c073be0 0x7f4c0c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.081+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 >> 172.21.15.6:0/4022907364 conn(0x7f4c0c0fc530 msgr2=0x7f4c0c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.082+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 shutdown_connections 2024-02-26T06:27:04.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.082+0000 7f4c0be00640 1 -- 172.21.15.6:0/4022907364 wait complete. 2024-02-26T06:27:04.100 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-26T06:27:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:04 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/764672926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.313+0000 7f3089200640 1 -- 172.21.15.6:0/2741597900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 msgr2=0x7f30841037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.313+0000 7f3089200640 1 --2- 172.21.15.6:0/2741597900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 0x7f30841037d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3074009ca0 tx=0x7f307402c100 comp rx=0 tx=0).stop 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 -- 172.21.15.6:0/2741597900 shutdown_connections 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 --2- 172.21.15.6:0/2741597900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 0x7f308410b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 --2- 172.21.15.6:0/2741597900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 --2- 172.21.15.6:0/2741597900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 0x7f30841037d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 -- 172.21.15.6:0/2741597900 >> 172.21.15.6:0/2741597900 conn(0x7f30840fe940 msgr2=0x7f3084100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 -- 172.21.15.6:0/2741597900 shutdown_connections 2024-02-26T06:27:04.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.314+0000 7f3089200640 1 -- 172.21.15.6:0/2741597900 wait complete. 2024-02-26T06:27:04.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 Processor -- start 2024-02-26T06:27:04.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 -- start start 2024-02-26T06:27:04.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 0x7f3084199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 0x7f30841a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a0f40 con 0x7f30841045a0 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a10b0 con 0x7f3084104f60 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.315+0000 7f3089200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a1270 con 0x7f30841033b0 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 0x7f3084199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38590/0 (socket says 172.21.15.6:38590) 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 -- 172.21.15.6:0/3299190304 learned_addr learned my addr 172.21.15.6:0/3299190304 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3082800640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 0x7f30841a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 -- 172.21.15.6:0/3299190304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 msgr2=0x7f3084199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 0x7f3084199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 -- 172.21.15.6:0/3299190304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 msgr2=0x7f30841a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 0x7f30841a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 -- 172.21.15.6:0/3299190304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009900 con 0x7f30841045a0 2024-02-26T06:27:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3082800640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 0x7f30841a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:04.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.316+0000 7f3081400640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084199990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e9f0 tx=0x7f306c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.317+0000 7f307aa00640 1 -- 172.21.15.6:0/3299190304 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c009700 con 0x7f30841045a0 2024-02-26T06:27:04.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.317+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841a1550 con 0x7f30841045a0 2024-02-26T06:27:04.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.317+0000 7f307aa00640 1 -- 172.21.15.6:0/3299190304 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f306c00cd80 con 0x7f30841045a0 2024-02-26T06:27:04.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.317+0000 7f307aa00640 1 -- 172.21.15.6:0/3299190304 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c018920 con 0x7f30841045a0 2024-02-26T06:27:04.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.317+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841a1a60 con 0x7f30841045a0 2024-02-26T06:27:04.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.320+0000 7f308bfde740 1 Processor -- start 2024-02-26T06:27:04.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.320+0000 7f308bfde740 1 -- start start 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f308bfde740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f308bfde740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f308bfde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f308bfde740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155301fb00 con 0x5615530b0f40 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f308bfde740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155301d030 con 0x5615530b0550 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f308bfde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561553018810 con 0x5615530b60b0 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f307aa00640 1 -- 172.21.15.6:0/3299190304 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f306c018a80 con 0x7f30841045a0 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3082800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38598/0 (socket says 172.21.15.6:38598) 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 -- 172.21.15.6:0/3959845623 learned_addr learned my addr 172.21.15.6:0/3959845623 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081400640 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 -- 172.21.15.6:0/3959845623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 msgr2=0x5615530b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530b8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 -- 172.21.15.6:0/3959845623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 msgr2=0x5615530b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081e00640 1 -- 172.21.15.6:0/3959845623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561552fe0490 con 0x5615530b0f40 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.321+0000 7f3081400640 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.322+0000 7f3081e00640 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530b5a40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f307402c710 tx=0x7f3074005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fe05a67655a72b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.322+0000 7f3073400640 1 -- 172.21.15.6:0/3959845623 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740327d0 con 0x5615530b0f40 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.322+0000 7f3073400640 1 -- 172.21.15.6:0/3959845623 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3074032930 con 0x5615530b0f40 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.322+0000 7f3073400640 1 -- 172.21.15.6:0/3959845623 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740042a0 con 0x5615530b0f40 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.322+0000 7f307aa00640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f30680a20c0 0x7f30680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f307aa00640 1 -- 172.21.15.6:0/3299190304 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f306c014070 con 0x7f30841045a0 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f3082800640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f30680a20c0 0x7f30680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 -- 172.21.15.6:0/3959845623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 msgr2=0x5615530b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530b5a40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f307402c710 tx=0x7f3074005ef0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 -- 172.21.15.6:0/3959845623 shutdown_connections 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530b5a40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 --2- 172.21.15.6:0/3959845623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 -- 172.21.15.6:0/3959845623 >> 172.21.15.6:0/3959845623 conn(0x5615530abac0 msgr2=0x5615530abec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 -- 172.21.15.6:0/3959845623 shutdown_connections 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f308bfde740 1 -- 172.21.15.6:0/3959845623 wait complete. 2024-02-26T06:27:04.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.323+0000 7f3082800640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f30680a20c0 0x7f30680a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5615530b17a0 tx=0x7f307c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.324+0000 7f308bfde740 1 -- start start 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.324+0000 7f308bfde740 1 Processor -- start 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 0x5615530ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0f40 0x5615530ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561552fbde70 con 0x5615530b0550 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561552fbdb10 con 0x5615530b0f40 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561552d60f20 con 0x5615530b60b0 2024-02-26T06:27:04.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f308bfde740 10 client.0 _ll_register_callbacks cb 0x561552fd1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 0x5615530ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 0x5615530ba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38608/0 (socket says 172.21.15.6:38608) 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 -- 172.21.15.6:0/3264754593 learned_addr learned my addr 172.21.15.6:0/3264754593 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081400640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0f40 0x5615530ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3082800640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 -- 172.21.15.6:0/3264754593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 msgr2=0x5615530c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 -- 172.21.15.6:0/3264754593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0f40 msgr2=0x5615530ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0f40 0x5615530ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081e00640 1 -- 172.21.15.6:0/3264754593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308410c0f0 con 0x5615530b0550 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.325+0000 7f3081400640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0f40 0x5615530ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f3081e00640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 0x5615530ba2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3074009430 tx=0x7f30740326d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd5cd185bc29df4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f3067e00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:04.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f3067e00640 1 -- 172.21.15.6:0/3264754593 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740327d0 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f3067e00640 1 -- 172.21.15.6:0/3264754593 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3074003970 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56155303cc10 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f3067e00640 1 -- 172.21.15.6:0/3264754593 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307404a650 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f308bfde740 10 client.14922 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.326+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561553016e90 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f3067e00640 1 -- 172.21.15.6:0/3264754593 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56155303cc10 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f3067e00640 1 -- 172.21.15.6:0/3264754593 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3074052030 con 0x5615530b0550 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f308bfde740 10 client.14922 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f308bfde740 10 client.14922 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f308bfde740 20 client.14922 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f308bfde740 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x5615530c6440 0x5615530ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f308bfde740 4 client.14922 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561552d186c0 con 0x5615530c6440 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.327+0000 7f3081400640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x5615530c6440 0x5615530ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.328+0000 7f3081400640 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x5615530c6440 0x5615530ca8c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebc28bb333bcbeb5 server_cookie=594b6b92e7dab3af in_seq=0 out_seq=0 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.328+0000 7f3067e00640 10 client.14922 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f3067e00640 1 -- 172.21.15.6:0/3264754593 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561552d186c0 con 0x5615530c6440 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f3067e00640 10 client.14922 handle_command_reply: tid=0 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 client.14922 shutdown 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 msgr2=0x5615530ba2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 0x5615530ba2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3074009430 tx=0x7f30740326d0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 shutdown_connections 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x5615530c6440 0x5615530ca8c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0f40 0x5615530ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 --2- 172.21.15.6:0/3264754593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0550 0x5615530ba2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f3067e00640 0 client.14922 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 >> 172.21.15.6:0/3264754593 conn(0x5615530abac0 msgr2=0x5615530aade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 shutdown_connections 2024-02-26T06:27:04.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 1 -- 172.21.15.6:0/3264754593 wait complete. 2024-02-26T06:27:04.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.329+0000 7f308bfde740 20 client.14922 trim_cache size 0 max 16384 2024-02-26T06:27:04.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.347+0000 7f308bfde740 1 Processor -- start 2024-02-26T06:27:04.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.347+0000 7f308bfde740 1 -- start start 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.347+0000 7f308bfde740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f308bfde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b0f40 0x5615530c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f308bfde740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b60b0 0x5615530cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f308bfde740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615530c2130 con 0x5615530b60b0 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f308bfde740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615530c8420 con 0x5615530b0550 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f308bfde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615530b8c00 con 0x5615530b0f40 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3082800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b60b0 0x5615530cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33424/0 (socket says 172.21.15.6:33424) 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 -- 172.21.15.6:0/2285385837 learned_addr learned my addr 172.21.15.6:0/2285385837 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b0f40 0x5615530c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 -- 172.21.15.6:0/2285385837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b0f40 msgr2=0x5615530c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b0f40 0x5615530c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 -- 172.21.15.6:0/2285385837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b60b0 msgr2=0x5615530cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b60b0 0x5615530cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.348+0000 7f3081e00640 1 -- 172.21.15.6:0/2285385837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561552fb3880 con 0x5615530b0550 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.349+0000 7f3081e00640 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530ce8e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f307403f6e0 tx=0x7f3074000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc46d80ead0cefa7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.349+0000 7f3072a00640 1 -- 172.21.15.6:0/2285385837 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307403f950 con 0x5615530b0550 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.349+0000 7f3072a00640 1 -- 172.21.15.6:0/2285385837 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f307404e070 con 0x5615530b0550 2024-02-26T06:27:04.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.349+0000 7f3072a00640 1 -- 172.21.15.6:0/2285385837 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074004610 con 0x5615530b0550 2024-02-26T06:27:04.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.349+0000 7f308bfde740 1 -- 172.21.15.6:0/2285385837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 msgr2=0x5615530ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.349+0000 7f308bfde740 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530ce8e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f307403f6e0 tx=0x7f3074000f40 comp rx=0 tx=0).stop 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 -- 172.21.15.6:0/2285385837 shutdown_connections 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b60b0 0x5615530cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b0f40 0x5615530c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 --2- 172.21.15.6:0/2285385837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530ce8e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 -- 172.21.15.6:0/2285385837 >> 172.21.15.6:0/2285385837 conn(0x5615530abac0 msgr2=0x5615530b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 -- 172.21.15.6:0/2285385837 shutdown_connections 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 -- 172.21.15.6:0/2285385837 wait complete. 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 -- start start 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.350+0000 7f308bfde740 1 Processor -- start 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530bdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615530bea20 con 0x5615530b0f40 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615530c1970 con 0x5615530b0550 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615530c1b30 con 0x5615530b60b0 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f308bfde740 10 client.0 _ll_register_callbacks cb 0x7f307c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530bdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530bdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38626/0 (socket says 172.21.15.6:38626) 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 -- 172.21.15.6:0/3471762804 learned_addr learned my addr 172.21.15.6:0/3471762804 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081e00640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3082800640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 -- 172.21.15.6:0/3471762804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 msgr2=0x5615530be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 -- 172.21.15.6:0/3471762804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 msgr2=0x5615530c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.351+0000 7f3081400640 1 -- 172.21.15.6:0/3471762804 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561553016e90 con 0x5615530b0f40 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f3081400640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530bdde0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f306c09ad80 tx=0x7f306c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3d9985055f25b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f3067e00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f3067e00640 1 -- 172.21.15.6:0/3471762804 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c09a240 con 0x5615530b0f40 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f3067e00640 1 -- 172.21.15.6:0/3471762804 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f306c09a3a0 con 0x5615530b0f40 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615530c7470 con 0x5615530b0f40 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f3067e00640 1 -- 172.21.15.6:0/3471762804 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306c09a560 con 0x5615530b0f40 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f308bfde740 10 client.14928 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:04.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561552fd94e0 con 0x5615530b0f40 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.352+0000 7f3067e00640 1 -- 172.21.15.6:0/3471762804 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5615530c7470 con 0x5615530b0f40 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f3067e00640 1 -- 172.21.15.6:0/3471762804 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f306c09fdb0 con 0x5615530b0f40 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f308bfde740 10 client.14928 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f308bfde740 10 client.14928 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f308bfde740 20 client.14928 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f308bfde740 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x561553129f80 0x5615531263e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f308bfde740 4 client.14928 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f3074032930 con 0x561553129f80 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f3081e00640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x561553129f80 0x5615531263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f3081e00640 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x561553129f80 0x5615531263e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6b28c732d7e7c7b server_cookie=e832005eb4a83720 in_seq=0 out_seq=0 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.353+0000 7f3067e00640 10 client.14928 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f3067e00640 1 -- 172.21.15.6:0/3471762804 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f3074032930 con 0x561553129f80 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f3067e00640 10 client.14928 handle_command_reply: tid=0 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 client.14928 shutdown 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 msgr2=0x5615530bdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530bdde0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f306c09ad80 tx=0x7f306c000bc0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 shutdown_connections 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x561553129f80 0x5615531263e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5615530b60b0 0x5615530be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615530b0f40 0x5615530bdde0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 --2- 172.21.15.6:0/3471762804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615530b0550 0x5615530c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f3067e00640 0 client.14928 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 >> 172.21.15.6:0/3471762804 conn(0x5615530abac0 msgr2=0x5615530ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 shutdown_connections 2024-02-26T06:27:04.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 1 -- 172.21.15.6:0/3471762804 wait complete. 2024-02-26T06:27:04.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.354+0000 7f308bfde740 20 client.14928 trim_cache size 0 max 16384 2024-02-26T06:27:04.361 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:04.361 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:04.361 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-02-26T06:27:04.361 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24569, 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 103.079785249, 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 116.066608456 2024-02-26T06:27:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:04.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f30680a20c0 msgr2=0x7f30680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f30680a20c0 0x7f30680a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5615530b17a0 tx=0x7f307c005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:04.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 msgr2=0x7f3084199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084199990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e9f0 tx=0x7f306c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 shutdown_connections 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f30680a20c0 0x7f30680a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3084104f60 0x7f30841a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30841045a0 0x7f3084199990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 --2- 172.21.15.6:0/3299190304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30841033b0 0x7f3084199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 >> 172.21.15.6:0/3299190304 conn(0x7f30840fe940 msgr2=0x7f3084107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 shutdown_connections 2024-02-26T06:27:04.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.355+0000 7f3089200640 1 -- 172.21.15.6:0/3299190304 wait complete. 2024-02-26T06:27:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:04 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/764672926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:04.373 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T06:27:04.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:04 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/764672926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 -- 172.21.15.6:0/2422292244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f8104d90 msgr2=0x7fe6f810b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/2422292244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f8104d90 0x7fe6f810b620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009be0 tx=0x7fe6f002bef0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 -- 172.21.15.6:0/2422292244 shutdown_connections 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/2422292244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f8104d90 0x7fe6f810b620 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/2422292244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/2422292244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f81031e0 0x7fe6f8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 -- 172.21.15.6:0/2422292244 >> 172.21.15.6:0/2422292244 conn(0x7fe6f80fe750 msgr2=0x7fe6f8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 -- 172.21.15.6:0/2422292244 shutdown_connections 2024-02-26T06:27:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.593+0000 7fe6fcc00640 1 -- 172.21.15.6:0/2422292244 wait complete. 2024-02-26T06:27:04.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 Processor -- start 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 -- start start 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f81031e0 0x7fe6f81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8104d90 0x7fe6f81a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81a0ce0 con 0x7fe6f81043d0 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81a0e50 con 0x7fe6f81031e0 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6fcc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81a1010 con 0x7fe6f8104d90 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38646/0 (socket says 172.21.15.6:38646) 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6f6a00640 1 -- 172.21.15.6:0/4223970202 learned_addr learned my addr 172.21.15.6:0/4223970202 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.595+0000 7fe6f7400640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f81031e0 0x7fe6f81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8104d90 0x7fe6f81a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f6a00640 1 -- 172.21.15.6:0/4223970202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8104d90 msgr2=0x7fe6f81a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8104d90 0x7fe6f81a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f6a00640 1 -- 172.21.15.6:0/4223970202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f81031e0 msgr2=0x7fe6f81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f81031e0 0x7fe6f81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f6a00640 1 -- 172.21.15.6:0/4223970202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e4009560 con 0x7fe6f81043d0 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f7400640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f81031e0 0x7fe6f81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.596+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8199730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e9c0 tx=0x7fe6e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.597+0000 7fe6efe00640 1 -- 172.21.15.6:0/4223970202 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400cde0 con 0x7fe6f81043d0 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.597+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f0009840 con 0x7fe6f81043d0 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.597+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f81a16c0 con 0x7fe6f81043d0 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.597+0000 7fe6efe00640 1 -- 172.21.15.6:0/4223970202 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6e40035f0 con 0x7fe6f81043d0 2024-02-26T06:27:04.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.597+0000 7fe6efe00640 1 -- 172.21.15.6:0/4223970202 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4010690 con 0x7fe6f81043d0 2024-02-26T06:27:04.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.600+0000 7fe701063740 1 Processor -- start 2024-02-26T06:27:04.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.600+0000 7fe701063740 1 -- start start 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe701063740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 0x55562400eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe701063740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624013c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe701063740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624014290 0x555624016720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe701063740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555623f7db00 con 0x555624014290 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe701063740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555623f7b030 con 0x55562400e730 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe701063740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555623f76810 con 0x55562400f120 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 0x55562400eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 0x55562400eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33472/0 (socket says 172.21.15.6:33472) 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 -- 172.21.15.6:0/2692357732 learned_addr learned my addr 172.21.15.6:0/2692357732 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7400640 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624013c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 -- 172.21.15.6:0/2692357732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 msgr2=0x555624013c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624013c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 -- 172.21.15.6:0/2692357732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624014290 msgr2=0x555624016720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:04.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624014290 0x555624016720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7e00640 1 -- 172.21.15.6:0/2692357732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555623f3e490 con 0x55562400e730 2024-02-26T06:27:04.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6f7400640 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624013c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:04.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.601+0000 7fe6efe00640 1 -- 172.21.15.6:0/4223970202 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe6e4004020 con 0x7fe6f81043d0 2024-02-26T06:27:04.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.602+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 0x55562400eb50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe6f002c500 tx=0x7fe6f002cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2ea4bfafca9e7b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.602+0000 7fe6e3e00640 1 -- 172.21.15.6:0/2692357732 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f002ca90 con 0x55562400e730 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.602+0000 7fe6e3e00640 1 -- 172.21.15.6:0/2692357732 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f0033a80 con 0x55562400e730 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.602+0000 7fe6e3e00640 1 -- 172.21.15.6:0/2692357732 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00042a0 con 0x55562400e730 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 -- 172.21.15.6:0/2692357732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 msgr2=0x55562400eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 0x55562400eb50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe6f002c500 tx=0x7fe6f002cd60 comp rx=0 tx=0).stop 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 -- 172.21.15.6:0/2692357732 shutdown_connections 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624014290 0x555624016720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624013c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 --2- 172.21.15.6:0/2692357732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400e730 0x55562400eb50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 -- 172.21.15.6:0/2692357732 >> 172.21.15.6:0/2692357732 conn(0x555624009ca0 msgr2=0x55562400a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 -- 172.21.15.6:0/2692357732 shutdown_connections 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe701063740 1 -- 172.21.15.6:0/2692357732 wait complete. 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe6efe00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe6dc0a1fd0 0x7fe6dc0a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.603+0000 7fe6efe00640 1 -- 172.21.15.6:0/4223970202 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe6e4014070 con 0x7fe6f81043d0 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.604+0000 7fe6f7400640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe6dc0a1fd0 0x7fe6dc0a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.604+0000 7fe701063740 1 -- start start 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.604+0000 7fe701063740 1 Processor -- start 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.604+0000 7fe6f7400640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe6dc0a1fd0 0x7fe6dc0a4490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55562400ac10 tx=0x7fe6e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.604+0000 7fe701063740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x5556240185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.604+0000 7fe701063740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 0x555624018af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe701063740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624014290 0x55562401fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe701063740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555623f1be70 con 0x55562400f120 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe701063740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555623f1bb10 con 0x555624014290 2024-02-26T06:27:04.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe701063740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555623cbef20 con 0x55562400e730 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe701063740 10 client.0 _ll_register_callbacks cb 0x555623f2f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 0x555624018af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x5556240185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x5556240185b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49348/0 (socket says 172.21.15.6:49348) 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 0x555624018af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38652/0 (socket says 172.21.15.6:38652) 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f7400640 1 -- 172.21.15.6:0/564598514 learned_addr learned my addr 172.21.15.6:0/564598514 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624014290 0x55562401fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 -- 172.21.15.6:0/564598514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 msgr2=0x5556240185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x5556240185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 -- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624014290 msgr2=0x55562401fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624014290 0x55562401fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 -- 172.21.15.6:0/564598514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f810bf20 con 0x55562400f120 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624014290 0x55562401fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 0x555624018af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400c6a0 tx=0x7fe6e409b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ec2336da47b652 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.605+0000 7fe6e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe6e0c00640 1 -- 172.21.15.6:0/564598514 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40b2070 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe6e0c00640 1 -- 172.21.15.6:0/564598514 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe6e40a3450 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555623f9ac10 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe6e0c00640 1 -- 172.21.15.6:0/564598514 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6e409ee80 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe701063740 10 client.14937 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555623f74e90 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe6e0c00640 1 -- 172.21.15.6:0/564598514 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6e407d080 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe6e0c00640 1 -- 172.21.15.6:0/564598514 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6e401f060 con 0x55562400f120 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe701063740 10 client.14937 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.606+0000 7fe701063740 10 client.14937 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.607+0000 7fe701063740 20 client.14937 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.607+0000 7fe701063740 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624024520 0x555624023e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.607+0000 7fe701063740 4 client.14937 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:04.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.607+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555623c766c0 con 0x555624024520 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.607+0000 7fe6f7400640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624024520 0x555624023e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.607+0000 7fe6f7400640 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624024520 0x555624023e20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463a37a1974d07d1 server_cookie=7bae6dee02b64c in_seq=0 out_seq=0 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.608+0000 7fe6e0c00640 10 client.14937 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.608+0000 7fe6e0c00640 1 -- 172.21.15.6:0/564598514 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555623c766c0 con 0x555624024520 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.608+0000 7fe6e0c00640 10 client.14937 handle_command_reply: tid=0 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.608+0000 7fe701063740 1 client.14937 shutdown 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 msgr2=0x555624018af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 0x555624018af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400c6a0 tx=0x7fe6e409b4f0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 shutdown_connections 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624024520 0x555624023e20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624014290 0x55562401fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400f120 0x555624018af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 --2- 172.21.15.6:0/564598514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x5556240185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe6e0c00640 0 client.14937 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:04.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 >> 172.21.15.6:0/564598514 conn(0x555624009ca0 msgr2=0x555624008de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 shutdown_connections 2024-02-26T06:27:04.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 1 -- 172.21.15.6:0/564598514 wait complete. 2024-02-26T06:27:04.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.609+0000 7fe701063740 20 client.14937 trim_cache size 0 max 16384 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.626+0000 7fe701063740 1 Processor -- start 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.626+0000 7fe701063740 1 -- start start 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe701063740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 0x555624017f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe701063740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400e730 0x55562401faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe701063740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624004c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe701063740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624024d50 con 0x55562400e730 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe701063740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624025a10 con 0x555624009ca0 2024-02-26T06:27:04.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe701063740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556240203f0 con 0x55562400f120 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 0x555624017f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624004c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 0x555624017f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:33498/0 (socket says 172.21.15.6:33498) 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 -- 172.21.15.6:0/3801364818 learned_addr learned my addr 172.21.15.6:0/3801364818 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624004c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49360/0 (socket says 172.21.15.6:49360) 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400e730 0x55562401faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 -- 172.21.15.6:0/3801364818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 msgr2=0x555624004c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624004c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 -- 172.21.15.6:0/3801364818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400e730 msgr2=0x55562401faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400e730 0x55562401faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.627+0000 7fe6f7400640 1 -- 172.21.15.6:0/3801364818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555623f11880 con 0x555624009ca0 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.628+0000 7fe6f7400640 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 0x555624017f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009460 tx=0x7fe6e801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301231aa34ad0dbf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.628+0000 7fe6e3400640 1 -- 172.21.15.6:0/3801364818 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8023050 con 0x555624009ca0 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.628+0000 7fe6e3400640 1 -- 172.21.15.6:0/3801364818 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6e8024040 con 0x555624009ca0 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.628+0000 7fe6e3400640 1 -- 172.21.15.6:0/3801364818 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8017070 con 0x555624009ca0 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.628+0000 7fe701063740 1 -- 172.21.15.6:0/3801364818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 msgr2=0x555624017f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.628+0000 7fe701063740 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 0x555624017f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009460 tx=0x7fe6e801b000 comp rx=0 tx=0).stop 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 -- 172.21.15.6:0/3801364818 shutdown_connections 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400f120 0x555624004c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55562400e730 0x55562401faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 --2- 172.21.15.6:0/3801364818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555624009ca0 0x555624017f90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 -- 172.21.15.6:0/3801364818 >> 172.21.15.6:0/3801364818 conn(0x5556240052d0 msgr2=0x555624017880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 -- 172.21.15.6:0/3801364818 shutdown_connections 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 -- 172.21.15.6:0/3801364818 wait complete. 2024-02-26T06:27:04.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 -- start start 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.629+0000 7fe701063740 1 Processor -- start 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 0x55562401cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x55562401d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400f120 0x55562401e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562401e720 con 0x555624009ca0 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562401e890 con 0x55562400f120 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562401ea50 con 0x55562400e730 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe701063740 10 client.0 _ll_register_callbacks cb 0x7fe6e4096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 0x55562401cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 0x55562401cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38662/0 (socket says 172.21.15.6:38662) 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 -- 172.21.15.6:0/3906139361 learned_addr learned my addr 172.21.15.6:0/3906139361 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7e00640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400f120 0x55562401e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x55562401d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 -- 172.21.15.6:0/3906139361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 msgr2=0x55562401d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x55562401d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 -- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400f120 msgr2=0x55562401e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400f120 0x55562401e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.630+0000 7fe6f7400640 1 -- 172.21.15.6:0/3906139361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555623f374e0 con 0x555624009ca0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe6f7400640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 0x55562401cd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6e802e040 tx=0x7fe6e802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50af97272d58016 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe6e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe6e0c00640 1 -- 172.21.15.6:0/3906139361 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8017040 con 0x555624009ca0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe6e0c00640 1 -- 172.21.15.6:0/3906139361 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe6e8005240 con 0x555624009ca0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555624025b50 con 0x555624009ca0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe6e0c00640 1 -- 172.21.15.6:0/3906139361 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6e8023030 con 0x555624009ca0 2024-02-26T06:27:04.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe701063740 10 client.14943 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555623f1dd70 con 0x555624009ca0 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.631+0000 7fe6e0c00640 1 -- 172.21.15.6:0/3906139361 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555624025b50 con 0x555624009ca0 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe6e0c00640 1 -- 172.21.15.6:0/3906139361 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6e8032070 con 0x555624009ca0 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe701063740 10 client.14943 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe701063740 10 client.14943 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe701063740 20 client.14943 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe701063740 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624014290 0x55562407c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe701063740 4 client.14943 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555624024e90 con 0x555624014290 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.632+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624014290 0x55562407c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe6f6a00640 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624014290 0x55562407c800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740312419b3c8aad server_cookie=f308378374cc21f0 in_seq=0 out_seq=0 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe6e0c00640 10 client.14943 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe6e0c00640 1 -- 172.21.15.6:0/3906139361 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x555624024e90 con 0x555624014290 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe6e0c00640 10 client.14943 handle_command_reply: tid=0 2024-02-26T06:27:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 client.14943 shutdown 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 msgr2=0x55562401cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 0x55562401cd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6e802e040 tx=0x7fe6e802a760 comp rx=0 tx=0).stop 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 shutdown_connections 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555624014290 0x55562407c800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55562400f120 0x55562401e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55562400e730 0x55562401d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 --2- 172.21.15.6:0/3906139361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555624009ca0 0x55562401cd30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe6e0c00640 0 client.14943 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 >> 172.21.15.6:0/3906139361 conn(0x5556240052d0 msgr2=0x555624008550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 shutdown_connections 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 1 -- 172.21.15.6:0/3906139361 wait complete. 2024-02-26T06:27:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.633+0000 7fe701063740 20 client.14943 trim_cache size 0 max 16384 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24587, 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 106.40187079, 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 113.61324742399999 2024-02-26T06:27:04.638 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:04.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe6dc0a1fd0 msgr2=0x7fe6dc0a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe6dc0a1fd0 0x7fe6dc0a4490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55562400ac10 tx=0x7fe6e8005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:04.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 msgr2=0x7fe6f8199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:04.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8199730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e9c0 tx=0x7fe6e400ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:04.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 shutdown_connections 2024-02-26T06:27:04.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8104d90 0x7fe6f81a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6f81043d0 0x7fe6f8199730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe6dc0a1fd0 0x7fe6dc0a4490 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 --2- 172.21.15.6:0/4223970202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f81031e0 0x7fe6f81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 >> 172.21.15.6:0/4223970202 conn(0x7fe6f80fe750 msgr2=0x7fe6f8107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 shutdown_connections 2024-02-26T06:27:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:04.634+0000 7fe6fcc00640 1 -- 172.21.15.6:0/4223970202 wait complete. 2024-02-26T06:27:04.652 INFO:tasks.cephadm:Setup complete, yielding 2024-02-26T06:27:04.652 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T06:27:04.661 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:27:04.662 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph orch status' 2024-02-26T06:27:05.113 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:05 smithi090 ceph-mon[35681]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:05.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:05 smithi006 ceph-mon[31790]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:05 smithi167 ceph-mon[36229]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:07.320 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:07 smithi006 ceph-mon[31790]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:07.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 -- 172.21.15.6:0/2500776952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238105060 msgr2=0x7fe238107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 --2- 172.21.15.6:0/2500776952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238105060 0x7fe238107450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2240098f0 tx=0x7fe224009c00 comp rx=0 tx=0).stop 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 -- 172.21.15.6:0/2500776952 shutdown_connections 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 --2- 172.21.15.6:0/2500776952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 0x7fe238109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 --2- 172.21.15.6:0/2500776952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238102730 0x7fe238104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 --2- 172.21.15.6:0/2500776952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238105060 0x7fe238107450 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 -- 172.21.15.6:0/2500776952 >> 172.21.15.6:0/2500776952 conn(0x7fe2380fc140 msgr2=0x7fe2380fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 -- 172.21.15.6:0/2500776952 shutdown_connections 2024-02-26T06:27:07.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.492+0000 7fe23f400700 1 -- 172.21.15.6:0/2500776952 wait complete. 2024-02-26T06:27:07.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.493+0000 7fe23f400700 1 Processor -- start 2024-02-26T06:27:07.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.493+0000 7fe23f400700 1 -- start start 2024-02-26T06:27:07.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.493+0000 7fe23f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238102730 0x7fe238199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:07.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.493+0000 7fe23f400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238105060 0x7fe23819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 0x7fe2381a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23f400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381a1ad0 con 0x7fe238107990 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23f400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381a1c10 con 0x7fe238102730 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23f400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381a1da0 con 0x7fe238105060 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe237e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238105060 0x7fe23819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 0x7fe2381a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238102730 0x7fe238199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe237e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238105060 0x7fe23819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:43000/0 (socket says 172.21.15.6:43000) 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe237e00700 1 -- 172.21.15.6:0/2763934243 learned_addr learned my addr 172.21.15.6:0/2763934243 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:07.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23ca00700 1 -- 172.21.15.6:0/2763934243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238105060 msgr2=0x7fe23819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23ca00700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238105060 0x7fe23819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23ca00700 1 -- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 msgr2=0x7fe2381a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23ca00700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 0x7fe2381a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23ca00700 1 -- 172.21.15.6:0/2763934243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0095f0 con 0x7fe238102730 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.494+0000 7fe23d400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 0x7fe2381a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.495+0000 7fe23ca00700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238102730 0x7fe238199f40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00eb90 tx=0x7fe22c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.495+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c009970 con 0x7fe238102730 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.495+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe22c004500 con 0x7fe238102730 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.495+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe224009520 con 0x7fe238102730 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.495+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c010430 con 0x7fe238102730 2024-02-26T06:27:07.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.495+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2381a2440 con 0x7fe238102730 2024-02-26T06:27:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:07 smithi167 ceph-mon[36229]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:07.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.496+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe238068930 con 0x7fe238102730 2024-02-26T06:27:07.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.498+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe22c00cc90 con 0x7fe238102730 2024-02-26T06:27:07.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.499+0000 7fe235600700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2200a21d0 0x7fe2200a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:07.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.500+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe22c014070 con 0x7fe238102730 2024-02-26T06:27:07.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.500+0000 7fe237e00700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2200a21d0 0x7fe2200a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:07.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.500+0000 7fe237e00700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2200a21d0 0x7fe2200a4690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe224005e90 tx=0x7fe224019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.504+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe22c097030 con 0x7fe238102730 2024-02-26T06:27:07.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:07 smithi090 ceph-mon[35681]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:07.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.749+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe238100e10 con 0x7fe2200a21d0 2024-02-26T06:27:07.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.751+0000 7fe235600700 1 -- 172.21.15.6:0/2763934243 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe238100e10 con 0x7fe2200a21d0 2024-02-26T06:27:07.751 INFO:teuthology.orchestra.run.smithi006.stdout:Backend: cephadm 2024-02-26T06:27:07.752 INFO:teuthology.orchestra.run.smithi006.stdout:Available: Yes 2024-02-26T06:27:07.752 INFO:teuthology.orchestra.run.smithi006.stdout:Paused: No 2024-02-26T06:27:07.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2200a21d0 msgr2=0x7fe2200a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:07.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2200a21d0 0x7fe2200a4690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe224005e90 tx=0x7fe224019040 comp rx=0 tx=0).stop 2024-02-26T06:27:07.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238102730 msgr2=0x7fe238199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238102730 0x7fe238199f40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00eb90 tx=0x7fe22c00eea0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 shutdown_connections 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2200a21d0 0x7fe2200a4690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe238107990 0x7fe2381a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe238102730 0x7fe238199f40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 --2- 172.21.15.6:0/2763934243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe238105060 0x7fe23819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 >> 172.21.15.6:0/2763934243 conn(0x7fe2380fc140 msgr2=0x7fe238100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:07.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 shutdown_connections 2024-02-26T06:27:07.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:07.754+0000 7fe23f400700 1 -- 172.21.15.6:0/2763934243 wait complete. 2024-02-26T06:27:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:08 smithi006 ceph-mon[31790]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:08 smithi167 ceph-mon[36229]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:08.567 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-26T06:27:08.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:08 smithi090 ceph-mon[35681]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:09.000 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:09.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:09 smithi006 ceph-mon[31790]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:27:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:09 smithi167 ceph-mon[36229]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:27:09.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:09 smithi090 ceph-mon[35681]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:27:11.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 -- 172.21.15.6:0/2822036764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180105150 msgr2=0x7f8180107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 --2- 172.21.15.6:0/2822036764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180105150 0x7f8180107540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8168009990 tx=0x7f8168009ca0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 -- 172.21.15.6:0/2822036764 shutdown_connections 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 --2- 172.21.15.6:0/2822036764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180107a80 0x7f8180109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 --2- 172.21.15.6:0/2822036764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180102820 0x7f8180104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 --2- 172.21.15.6:0/2822036764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180105150 0x7f8180107540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 -- 172.21.15.6:0/2822036764 >> 172.21.15.6:0/2822036764 conn(0x7f81800fc170 msgr2=0x7f81800fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 -- 172.21.15.6:0/2822036764 shutdown_connections 2024-02-26T06:27:11.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.288+0000 7f8185400700 1 -- 172.21.15.6:0/2822036764 wait complete. 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.289+0000 7f8185400700 1 Processor -- start 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.289+0000 7f8185400700 1 -- start start 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.289+0000 7f8185400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 0x7f8180102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8185400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180105150 0x7f81801007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8185400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180107a80 0x7f8180100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8185400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180101490 con 0x7f8180102820 2024-02-26T06:27:11.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8185400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180101600 con 0x7f8180107a80 2024-02-26T06:27:11.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8185400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81801a1c70 con 0x7f8180105150 2024-02-26T06:27:11.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 0x7f8180102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 0x7f8180102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51178/0 (socket says 172.21.15.6:51178) 2024-02-26T06:27:11.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 -- 172.21.15.6:0/3865215269 learned_addr learned my addr 172.21.15.6:0/3865215269 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:11.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8175e00700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180105150 0x7f81801007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817f200700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180107a80 0x7f8180100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 -- 172.21.15.6:0/3865215269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180105150 msgr2=0x7f81801007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180105150 0x7f81801007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 -- 172.21.15.6:0/3865215269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180107a80 msgr2=0x7f8180100d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180107a80 0x7f8180100d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f817e800700 1 -- 172.21.15.6:0/3865215269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680095f0 con 0x7f8180102820 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.290+0000 7f8175e00700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180105150 0x7f81801007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.291+0000 7f817e800700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 0x7f8180102180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f817000e9a0 tx=0x7f817000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.291+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170009790 con 0x7f8180102820 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.291+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8170010430 con 0x7f8180102820 2024-02-26T06:27:11.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.291+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170018680 con 0x7f8180102820 2024-02-26T06:27:11.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.291+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81801a1f50 con 0x7f8180102820 2024-02-26T06:27:11.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.291+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81801a22b0 con 0x7f8180102820 2024-02-26T06:27:11.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.293+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8180068970 con 0x7f8180102820 2024-02-26T06:27:11.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.295+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8170010910 con 0x7f8180102820 2024-02-26T06:27:11.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.296+0000 7f8177e00700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f81640a2150 0x7f81640a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:11.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.296+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8170014070 con 0x7f8180102820 2024-02-26T06:27:11.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.296+0000 7f8175e00700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f81640a2150 0x7f81640a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:11.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.297+0000 7f8175e00700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f81640a2150 0x7f81640a4610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f816800b650 tx=0x7f8168005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:11.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.301+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8170052d20 con 0x7f8180102820 2024-02-26T06:27:11.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:11 smithi006 ceph-mon[31790]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-02-26T06:27:11.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:11 smithi167 ceph-mon[36229]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-02-26T06:27:11.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.538+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f818002dab0 con 0x7f81640a2150 2024-02-26T06:27:11.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.580+0000 7f8177e00700 1 -- 172.21.15.6:0/3865215269 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f818002dab0 con 0x7f81640a2150 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.a smithi006 running (2m) 24s ago 2m 16.3M - 19.0.0-1514-g25296a8f a4519cb49bed 1fe397810ee4 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.b smithi090 running (2m) 52s ago 2m 28.4M - 19.0.0-1514-g25296a8f a4519cb49bed 9a03e2fa1b16 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.c smithi167 running (2m) 52s ago 2m 17.2M - 19.0.0-1514-g25296a8f a4519cb49bed a5841f2d50c2 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.d smithi006 running (2m) 24s ago 2m 15.6M - 19.0.0-1514-g25296a8f a4519cb49bed 16c20d018452 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.e smithi090 running (2m) 52s ago 2m 18.2M - 19.0.0-1514-g25296a8f a4519cb49bed 0eea3ad0d4b9 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.f smithi167 running (2m) 52s ago 2m 17.9M - 19.0.0-1514-g25296a8f a4519cb49bed bee6b06b91a6 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.g smithi006 running (2m) 24s ago 2m 12.2M - 19.0.0-1514-g25296a8f a4519cb49bed e1e5e0342d40 2024-02-26T06:27:11.581 INFO:teuthology.orchestra.run.smithi006.stdout:mds.h smithi090 running (2m) 52s ago 2m 17.3M - 19.0.0-1514-g25296a8f a4519cb49bed a4ba64c7d6e9 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mds.i smithi167 running (2m) 52s ago 2m 19.7M - 19.0.0-1514-g25296a8f a4519cb49bed 688601f53c49 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mds.j smithi006 running (2m) 24s ago 2m 12.4M - 19.0.0-1514-g25296a8f a4519cb49bed be5dd0fd6292 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mds.k smithi090 running (2m) 52s ago 2m 16.7M - 19.0.0-1514-g25296a8f a4519cb49bed 4d0a86465023 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mds.l smithi167 running (2m) 52s ago 2m 21.3M - 19.0.0-1514-g25296a8f a4519cb49bed bf954bc3704b 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.x smithi006 *:9283,8765 running (10m) 24s ago 10m 514M - 19.0.0-1514-g25296a8f a4519cb49bed 521a74fa3287 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.y smithi090 *:8443,8765 running (8m) 52s ago 8m 451M - 19.0.0-1514-g25296a8f a4519cb49bed 8bed9c8b972d 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.z smithi167 *:8443,8765 running (8m) 52s ago 8m 450M - 19.0.0-1514-g25296a8f a4519cb49bed f4db46f9f89b 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mon.a smithi006 running (11m) 24s ago 11m 47.7M 2048M 19.0.0-1514-g25296a8f a4519cb49bed daf24c1bbbd0 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mon.b smithi090 running (8m) 52s ago 8m 39.4M 2048M 19.0.0-1514-g25296a8f a4519cb49bed 09d413e3f114 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:mon.c smithi167 running (9m) 52s ago 9m 40.9M 2048M 19.0.0-1514-g25296a8f a4519cb49bed fd386e6b9e0d 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (8m) 24s ago 8m 55.2M 4273M 19.0.0-1514-g25296a8f a4519cb49bed 5b81f99350a9 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi090 running (7m) 52s ago 7m 57.3M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 556178e51b85 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi167 running (7m) 52s ago 7m 56.2M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 033f9dfc0e40 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi006 running (6m) 24s ago 6m 45.6M 4273M 19.0.0-1514-g25296a8f a4519cb49bed 5587f0477a57 2024-02-26T06:27:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi090 running (6m) 52s ago 6m 52.4M 4270M 19.0.0-1514-g25296a8f a4519cb49bed f315cd2920fb 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi167 running (5m) 52s ago 5m 48.7M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 6f64f59f7a3f 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 smithi006 running (5m) 24s ago 5m 51.4M 4273M 19.0.0-1514-g25296a8f a4519cb49bed b4f960800e7b 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 smithi090 running (4m) 52s ago 4m 53.9M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 8247d7822dff 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 smithi167 running (4m) 52s ago 4m 55.4M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 4d65b9a62d2c 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 smithi006 running (3m) 24s ago 3m 43.7M 4273M 19.0.0-1514-g25296a8f a4519cb49bed d8ded68e1c37 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 smithi090 running (3m) 52s ago 3m 45.7M 4270M 19.0.0-1514-g25296a8f a4519cb49bed ea2a6e76e2f9 2024-02-26T06:27:11.583 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 smithi167 running (3m) 52s ago 3m 51.6M 4270M 19.0.0-1514-g25296a8f a4519cb49bed a2d1f78c06ec 2024-02-26T06:27:11.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f81640a2150 msgr2=0x7f81640a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f81640a2150 0x7f81640a4610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f816800b650 tx=0x7f8168005fb0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 msgr2=0x7f8180102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 0x7f8180102180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f817000e9a0 tx=0x7f817000ecb0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 shutdown_connections 2024-02-26T06:27:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f81640a2150 0x7f81640a4610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8180107a80 0x7f8180100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8180102820 0x7f8180102180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 --2- 172.21.15.6:0/3865215269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180105150 0x7f81801007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 >> 172.21.15.6:0/3865215269 conn(0x7f81800fc170 msgr2=0x7f8180103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 shutdown_connections 2024-02-26T06:27:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:11.584+0000 7f8185400700 1 -- 172.21.15.6:0/3865215269 wait complete. 2024-02-26T06:27:11.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:11 smithi090 ceph-mon[35681]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-02-26T06:27:12.392 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-26T06:27:12.489 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:12 smithi006 ceph-mon[31790]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:12 smithi167 ceph-mon[36229]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:12 smithi090 ceph-mon[35681]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:12.871 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:13.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:13 smithi006 ceph-mon[31790]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-02-26T06:27:13.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:13 smithi167 ceph-mon[36229]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-02-26T06:27:13.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:13 smithi090 ceph-mon[35681]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-02-26T06:27:15.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 -- 172.21.15.6:0/3672548859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0102ed0 msgr2=0x7f5ce0103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:15.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3672548859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0102ed0 0x7f5ce0103330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0099c0 tx=0x7f5ccc009cd0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 -- 172.21.15.6:0/3672548859 shutdown_connections 2024-02-26T06:27:15.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3672548859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0103870 0x7f5ce010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3672548859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ce0101cd0 0x7f5ce01020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3672548859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0102ed0 0x7f5ce0103330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 -- 172.21.15.6:0/3672548859 >> 172.21.15.6:0/3672548859 conn(0x7f5ce00fd270 msgr2=0x7f5ce00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:15.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 -- 172.21.15.6:0/3672548859 shutdown_connections 2024-02-26T06:27:15.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.095+0000 7f5ce7800700 1 -- 172.21.15.6:0/3672548859 wait complete. 2024-02-26T06:27:15.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.097+0000 7f5ce7800700 1 Processor -- start 2024-02-26T06:27:15.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.097+0000 7f5ce7800700 1 -- start start 2024-02-26T06:27:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.097+0000 7f5ce7800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 0x7f5ce0197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.097+0000 7f5ce7800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0102ed0 0x7f5ce01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.097+0000 7f5ce7800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ce0103870 0x7f5ce019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce7800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce019f7e0 con 0x7f5ce0103870 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce7800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce019f920 con 0x7f5ce0101cd0 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce7800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce019fae0 con 0x7f5ce0102ed0 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 0x7f5ce0197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 0x7f5ce0197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:39752/0 (socket says 172.21.15.6:39752) 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce4e00700 1 -- 172.21.15.6:0/3371631177 learned_addr learned my addr 172.21.15.6:0/3371631177 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce5800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ce0103870 0x7f5ce019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5cdfe00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0102ed0 0x7f5ce01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5cdfe00700 1 -- 172.21.15.6:0/3371631177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 msgr2=0x7f5ce0197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5cdfe00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 0x7f5ce0197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5cdfe00700 1 -- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ce0103870 msgr2=0x7f5ce019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5cdfe00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ce0103870 0x7f5ce019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5cdfe00700 1 -- 172.21.15.6:0/3371631177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc0095f0 con 0x7f5ce0102ed0 2024-02-26T06:27:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.098+0000 7f5ce4e00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 0x7f5ce0197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.099+0000 7f5cdfe00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0102ed0 0x7f5ce01981a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009fd0 tx=0x7f5ccc0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.099+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc01d070 con 0x7f5ce0102ed0 2024-02-26T06:27:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.099+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce019fd60 con 0x7f5ce0102ed0 2024-02-26T06:27:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.099+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ccc003e90 con 0x7f5ce0102ed0 2024-02-26T06:27:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.099+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00fb80 con 0x7f5ce0102ed0 2024-02-26T06:27:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.099+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce01a0180 con 0x7f5ce0102ed0 2024-02-26T06:27:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.100+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce0109090 con 0x7f5ce0102ed0 2024-02-26T06:27:15.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.102+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5ccc00b9a0 con 0x7f5ce0102ed0 2024-02-26T06:27:15.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.104+0000 7f5cdd600700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5cc80a21d0 0x7f5cc80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:15.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.104+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5ccc0ceda0 con 0x7f5ce0102ed0 2024-02-26T06:27:15.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.104+0000 7f5ce4e00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5cc80a21d0 0x7f5cc80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:15.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.105+0000 7f5ce4e00700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5cc80a21d0 0x7f5cc80a4690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0198b40 tx=0x7f5cd400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:15.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.106+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ccc097d90 con 0x7f5ce0102ed0 2024-02-26T06:27:15.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.344+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ce002d8e0 con 0x7f5cc80a21d0 2024-02-26T06:27:15.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.362+0000 7f5cdd600700 1 -- 172.21.15.6:0/3371631177 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f5ce002d8e0 con 0x7f5cc80a21d0 2024-02-26T06:27:15.362 INFO:teuthology.orchestra.run.smithi006.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-26T06:27:15.362 INFO:teuthology.orchestra.run.smithi006.stdout:mds.all 12/12 56s ago 2m smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-02-26T06:27:15.362 INFO:teuthology.orchestra.run.smithi006.stdout:mgr 3/3 56s ago 8m smithi006=x;smithi090=y;smithi167=z;count:3 2024-02-26T06:27:15.362 INFO:teuthology.orchestra.run.smithi006.stdout:mon 3/3 56s ago 9m smithi006:172.21.15.6=a;smithi090:172.21.15.90=b;smithi167:172.21.15.167=c;count:3 2024-02-26T06:27:15.362 INFO:teuthology.orchestra.run.smithi006.stdout:osd 12 56s ago - 2024-02-26T06:27:15.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.364+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5cc80a21d0 msgr2=0x7f5cc80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:15.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.364+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5cc80a21d0 0x7f5cc80a4690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0198b40 tx=0x7f5cd400b040 comp rx=0 tx=0).stop 2024-02-26T06:27:15.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.364+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0102ed0 msgr2=0x7f5ce01981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:15.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.364+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0102ed0 0x7f5ce01981a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009fd0 tx=0x7f5ccc0055f0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 shutdown_connections 2024-02-26T06:27:15.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5cc80a21d0 0x7f5cc80a4690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ce0103870 0x7f5ce019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce0101cd0 0x7f5ce0197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 --2- 172.21.15.6:0/3371631177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce0102ed0 0x7f5ce01981a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 >> 172.21.15.6:0/3371631177 conn(0x7f5ce00fd270 msgr2=0x7f5ce01061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 shutdown_connections 2024-02-26T06:27:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:15.365+0000 7f5ce7800700 1 -- 172.21.15.6:0/3371631177 wait complete. 2024-02-26T06:27:15.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:15 smithi167 ceph-mon[36229]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:27:15.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:15 smithi090 ceph-mon[35681]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:27:15.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:15 smithi006 ceph-mon[31790]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:27:16.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-26T06:27:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:16 smithi006 ceph-mon[31790]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:16.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:16 smithi090 ceph-mon[35681]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:16.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:16 smithi167 ceph-mon[36229]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:17.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:17 smithi006 ceph-mon[31790]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:17 smithi090 ceph-mon[35681]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:17.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:17 smithi167 ceph-mon[36229]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 -- 172.21.15.6:0/3125950363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 msgr2=0x7fa1f8104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 --2- 172.21.15.6:0/3125950363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8104b20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009990 tx=0x7fa1e8009ca0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 -- 172.21.15.6:0/3125950363 shutdown_connections 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 --2- 172.21.15.6:0/3125950363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 0x7fa1f8109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 --2- 172.21.15.6:0/3125950363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8104b20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 --2- 172.21.15.6:0/3125950363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f8107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 -- 172.21.15.6:0/3125950363 >> 172.21.15.6:0/3125950363 conn(0x7fa1f80fc140 msgr2=0x7fa1f80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 -- 172.21.15.6:0/3125950363 shutdown_connections 2024-02-26T06:27:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.715+0000 7fa1fee00700 1 -- 172.21.15.6:0/3125950363 wait complete. 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.716+0000 7fa1fee00700 1 Processor -- start 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.716+0000 7fa1fee00700 1 -- start start 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fee00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fee00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 0x7fa1f81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fee00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81a1ad0 con 0x7fa1f8102730 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81a1c10 con 0x7fa1f8107990 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fee00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81a1da0 con 0x7fa1f8105060 2024-02-26T06:27:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1f7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1f7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1f7400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:38422/0 (socket says 172.21.15.6:38422) 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1f7400700 1 -- 172.21.15.6:0/285684502 learned_addr learned my addr 172.21.15.6:0/285684502 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1f7e00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38830/0 (socket says 172.21.15.6:38830) 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.717+0000 7fa1fce00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 0x7fa1f81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7400700 1 -- 172.21.15.6:0/285684502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 msgr2=0x7fa1f81a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7400700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 0x7fa1f81a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7400700 1 -- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 msgr2=0x7fa1f8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7400700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7400700 1 -- 172.21.15.6:0/285684502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009c40 con 0x7fa1f8105060 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7e00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1fce00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 0x7fa1f81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f7400700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f819a480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400e9d0 tx=0x7fa1e400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4009790 con 0x7fa1f8105060 2024-02-26T06:27:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80095f0 con 0x7fa1f8105060 2024-02-26T06:27:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1e400ee60 con 0x7fa1f8105060 2024-02-26T06:27:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.718+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4018680 con 0x7fa1f8105060 2024-02-26T06:27:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.719+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f81a2440 con 0x7fa1f8105060 2024-02-26T06:27:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.720+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f8109560 con 0x7fa1f8105060 2024-02-26T06:27:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.722+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fa1e400cbd0 con 0x7fa1f8105060 2024-02-26T06:27:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.723+0000 7fa1f4c00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa1dc0a2150 0x7fa1dc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:18.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.723+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa1e4014070 con 0x7fa1f8105060 2024-02-26T06:27:18.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.723+0000 7fa1f7e00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa1dc0a2150 0x7fa1dc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:18.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.724+0000 7fa1f7e00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa1dc0a2150 0x7fa1dc0a4610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800b2e0 tx=0x7fa1e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.725+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1e4096dd0 con 0x7fa1f8105060 2024-02-26T06:27:18.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.957+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1f802d870 con 0x7fa1dc0a2150 2024-02-26T06:27:18.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.959+0000 7fa1f4c00700 1 -- 172.21.15.6:0/285684502 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fa1f802d870 con 0x7fa1dc0a2150 2024-02-26T06:27:18.959 INFO:teuthology.orchestra.run.smithi006.stdout:HOST ADDR LABELS STATUS 2024-02-26T06:27:18.960 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 172.21.15.6 2024-02-26T06:27:18.960 INFO:teuthology.orchestra.run.smithi006.stdout:smithi090 172.21.15.90 2024-02-26T06:27:18.960 INFO:teuthology.orchestra.run.smithi006.stdout:smithi167 172.21.15.167 2024-02-26T06:27:18.960 INFO:teuthology.orchestra.run.smithi006.stdout:3 hosts in cluster 2024-02-26T06:27:18.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa1dc0a2150 msgr2=0x7fa1dc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:18.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa1dc0a2150 0x7fa1dc0a4610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800b2e0 tx=0x7fa1e8005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 msgr2=0x7fa1f819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f819a480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400e9d0 tx=0x7fa1e400ece0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 shutdown_connections 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fa1dc0a2150 0x7fa1dc0a4610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1f8107990 0x7fa1f81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1f8102730 0x7fa1f8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 --2- 172.21.15.6:0/285684502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f8105060 0x7fa1f819a480 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 >> 172.21.15.6:0/285684502 conn(0x7fa1f80fc140 msgr2=0x7fa1f8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 shutdown_connections 2024-02-26T06:27:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:18.962+0000 7fa1fee00700 1 -- 172.21.15.6:0/285684502 wait complete. 2024-02-26T06:27:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:19 smithi090 ceph-mon[35681]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:19 smithi090 ceph-mon[35681]: from='client.25090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:19.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:19 smithi006 ceph-mon[31790]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:19.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:19 smithi006 ceph-mon[31790]: from='client.25090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:19 smithi167 ceph-mon[36229]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:19 smithi167 ceph-mon[36229]: from='client.25090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:19.769 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-26T06:27:20.254 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:21 smithi090 ceph-mon[35681]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:21.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:21 smithi006 ceph-mon[31790]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:21.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:21 smithi167 ceph-mon[36229]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.583+0000 7ff1e6a00700 1 -- 172.21.15.6:0/3147238987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0074650 msgr2=0x7ff1e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.583+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/3147238987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0074650 0x7ff1e0074ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009990 tx=0x7ff1cc009ca0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 -- 172.21.15.6:0/3147238987 shutdown_connections 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/3147238987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1e0109ba0 0x7ff1e010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/3147238987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e0075c20 0x7ff1e0074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/3147238987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0074650 0x7ff1e0074ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 -- 172.21.15.6:0/3147238987 >> 172.21.15.6:0/3147238987 conn(0x7ff1e00fd280 msgr2=0x7ff1e00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 -- 172.21.15.6:0/3147238987 shutdown_connections 2024-02-26T06:27:22.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.584+0000 7ff1e6a00700 1 -- 172.21.15.6:0/3147238987 wait complete. 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.585+0000 7ff1e6a00700 1 Processor -- start 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.585+0000 7ff1e6a00700 1 -- start start 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1e0074650 0x7ff1e0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 0x7ff1e019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e6a00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e0109ba0 0x7ff1e01a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e6a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a19a0 con 0x7ff1e0075c20 2024-02-26T06:27:22.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e6a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a1ae0 con 0x7ff1e0074650 2024-02-26T06:27:22.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e6a00700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a1c70 con 0x7ff1e0109ba0 2024-02-26T06:27:22.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 0x7ff1e019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:22.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 0x7ff1e019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38842/0 (socket says 172.21.15.6:38842) 2024-02-26T06:27:22.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 -- 172.21.15.6:0/1090208161 learned_addr learned my addr 172.21.15.6:0/1090208161 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:22.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1e0074650 0x7ff1e0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:22.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1e4a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e0109ba0 0x7ff1e01a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 -- 172.21.15.6:0/1090208161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e0109ba0 msgr2=0x7ff1e01a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e0109ba0 0x7ff1e01a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 -- 172.21.15.6:0/1090208161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1e0074650 msgr2=0x7ff1e0199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1e0074650 0x7ff1e0199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.586+0000 7ff1df400700 1 -- 172.21.15.6:0/1090208161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009520 con 0x7ff1e0075c20 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.587+0000 7ff1df400700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 0x7ff1e019a350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0098e0 tx=0x7ff1cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.587+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc01d070 con 0x7ff1e0075c20 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.587+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0095f0 con 0x7ff1e0075c20 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.587+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc021410 con 0x7ff1e0075c20 2024-02-26T06:27:22.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.587+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc017650 con 0x7ff1e0075c20 2024-02-26T06:27:22.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.587+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e01a2170 con 0x7ff1e0075c20 2024-02-26T06:27:22.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.588+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e01089b0 con 0x7ff1e0075c20 2024-02-26T06:27:22.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.590+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff1cc021580 con 0x7ff1e0075c20 2024-02-26T06:27:22.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.592+0000 7ff1dcc00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1c40a2150 0x7ff1c40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:22.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.592+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff1cc0cf660 con 0x7ff1e0075c20 2024-02-26T06:27:22.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.592+0000 7ff1dfe00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1c40a2150 0x7ff1c40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:22.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.593+0000 7ff1dfe00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1c40a2150 0x7ff1c40a4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009df0 tx=0x7ff1d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.595+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1cc098650 con 0x7ff1e0075c20 2024-02-26T06:27:22.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.834+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e002d940 con 0x7ff1c40a2150 2024-02-26T06:27:22.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.839+0000 7ff1dcc00700 1 -- 172.21.15.6:0/1090208161 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7ff1e002d940 con 0x7ff1c40a2150 2024-02-26T06:27:22.839 INFO:teuthology.orchestra.run.smithi006.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-26T06:27:22.839 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330000H400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-26T06:27:22.840 INFO:teuthology.orchestra.run.smithi006.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-26T06:27:22.840 INFO:teuthology.orchestra.run.smithi006.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-26T06:27:22.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1c40a2150 msgr2=0x7ff1c40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:22.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1c40a2150 0x7ff1c40a4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009df0 tx=0x7ff1d0009260 comp rx=0 tx=0).stop 2024-02-26T06:27:22.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 msgr2=0x7ff1e019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:22.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 0x7ff1e019a350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0098e0 tx=0x7ff1cc005740 comp rx=0 tx=0).stop 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 shutdown_connections 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1c40a2150 0x7ff1c40a4610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e0109ba0 0x7ff1e01a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1e0074650 0x7ff1e0199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 --2- 172.21.15.6:0/1090208161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e0075c20 0x7ff1e019a350 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.842+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 >> 172.21.15.6:0/1090208161 conn(0x7ff1e00fd280 msgr2=0x7ff1e0105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.843+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 shutdown_connections 2024-02-26T06:27:22.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:22.843+0000 7ff1e6a00700 1 -- 172.21.15.6:0/1090208161 wait complete. 2024-02-26T06:27:23.610 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:23 smithi006 ceph-mon[31790]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:23.610 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:23 smithi006 ceph-mon[31790]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:23 smithi090 ceph-mon[35681]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:23 smithi090 ceph-mon[35681]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:23.692 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T06:27:23.702 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:27:23.703 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-26T06:27:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:23 smithi167 ceph-mon[36229]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:23 smithi167 ceph-mon[36229]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:24.207 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:25.559 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:25 smithi006 ceph-mon[31790]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:25 smithi090 ceph-mon[35681]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:25.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:25 smithi167 ceph-mon[36229]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 -- 172.21.15.6:0/2114649094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac074670 msgr2=0x7fb9ac074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 --2- 172.21.15.6:0/2114649094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac074670 0x7fb9ac074af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb994009990 tx=0x7fb994009ca0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 -- 172.21.15.6:0/2114649094 shutdown_connections 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 --2- 172.21.15.6:0/2114649094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9ac109b80 0x7fb9ac10c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 --2- 172.21.15.6:0/2114649094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac0740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 --2- 172.21.15.6:0/2114649094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac074670 0x7fb9ac074af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 -- 172.21.15.6:0/2114649094 >> 172.21.15.6:0/2114649094 conn(0x7fb9ac0fd280 msgr2=0x7fb9ac0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 -- 172.21.15.6:0/2114649094 shutdown_connections 2024-02-26T06:27:26.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.393+0000 7fb9b2000700 1 -- 172.21.15.6:0/2114649094 wait complete. 2024-02-26T06:27:26.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.394+0000 7fb9b2000700 1 Processor -- start 2024-02-26T06:27:26.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.394+0000 7fb9b2000700 1 -- start start 2024-02-26T06:27:26.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.394+0000 7fb9b2000700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9ac074670 0x7fb9ac199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9b2000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac109b80 0x7fb9ac1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9b2000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a19b0 con 0x7fb9ac109b80 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9b2000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a1af0 con 0x7fb9ac075c40 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9b2000700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a1c80 con 0x7fb9ac074670 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac109b80 0x7fb9ac1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac109b80 0x7fb9ac1a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38850/0 (socket says 172.21.15.6:38850) 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9ab400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9ac074670 0x7fb9ac199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:59422/0 (socket says 172.21.15.6:59422) 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9abe00700 1 -- 172.21.15.6:0/3628488374 learned_addr learned my addr 172.21.15.6:0/3628488374 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 -- 172.21.15.6:0/3628488374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9ac074670 msgr2=0x7fb9ac199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9ac074670 0x7fb9ac199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 -- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac109b80 msgr2=0x7fb9ac1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac109b80 0x7fb9ac1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.395+0000 7fb9a3e00700 1 -- 172.21.15.6:0/3628488374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9940095f0 con 0x7fb9ac075c40 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.396+0000 7fb9a3e00700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac19a360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb994000c00 tx=0x7fb99400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.396+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99401d070 con 0x7fb9ac075c40 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.396+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb99400be60 con 0x7fb9ac075c40 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.396+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940173f0 con 0x7fb9ac075c40 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.396+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac1a1f00 con 0x7fb9ac075c40 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.396+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac1a2310 con 0x7fb9ac075c40 2024-02-26T06:27:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.397+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac108990 con 0x7fb9ac075c40 2024-02-26T06:27:26.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.399+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb994021410 con 0x7fb9ac075c40 2024-02-26T06:27:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.400+0000 7fb9a8c00700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9900a21d0 0x7fb9900a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.401+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb9940cee00 con 0x7fb9ac075c40 2024-02-26T06:27:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.401+0000 7fb9ab400700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9900a21d0 0x7fb9900a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.401+0000 7fb9ab400700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9900a21d0 0x7fb9900a4690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009c60 tx=0x7fb99c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:26.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.405+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb994097f10 con 0x7fb9ac075c40 2024-02-26T06:27:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.680+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb9ac0689b0 con 0x7fb9ac075c40 2024-02-26T06:27:26.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.681+0000 7fb9a8c00700 1 -- 172.21.15.6:0/3628488374 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2945 (secure 0 0 0) 0x7fb994026ba0 con 0x7fb9ac075c40 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout:e26 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-02-26T06:27:26.683 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 26 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-02-26T06:25:17.657061+0000 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-02-26T06:25:24.777181+0000 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 300 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-02-26T06:27:26.684 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 5 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:in 0,1,2,3,4 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=24587,1=24569,2=24544,3=24530,4=24520} 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3] 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 0 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b{0:24587} state up:active seq 3 addr [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.f{0:24508} state up:standby-replay seq 1 addr [v2:172.21.15.167:6834/2489701568,v1:172.21.15.167:6835/2489701568] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.685 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a{1:24569} state up:active seq 5 addr [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.e{1:24503} state up:standby-replay seq 1 addr [v2:172.21.15.90:6834/679395830,v1:172.21.15.90:6835/679395830] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.i{2:24544} state up:active seq 7 addr [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d{2:24481} state up:standby-replay seq 1 addr [v2:172.21.15.6:6836/234881158,v1:172.21.15.6:6837/234881158] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.h{3:24530} state up:active seq 8 addr [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.k{3:24458} state up:standby-replay seq 1 addr [v2:172.21.15.90:6832/707593181,v1:172.21.15.90:6833/707593181] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c{4:24520} state up:active seq 9 addr [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.l{4:24445} state up:standby-replay seq 1 addr [v2:172.21.15.167:6832/66780761,v1:172.21.15.167:6833/66780761] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.j{-1:14376} state up:standby seq 1 addr [v2:172.21.15.6:6834/1064493742,v1:172.21.15.6:6835/1064493742] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.686 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.g{-1:14439} state up:standby seq 1 addr [v2:172.21.15.6:6838/1183641688,v1:172.21.15.6:6839/1183641688] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T06:27:26.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9900a21d0 msgr2=0x7fb9900a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:26.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9900a21d0 0x7fb9900a4690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009c60 tx=0x7fb99c009260 comp rx=0 tx=0).stop 2024-02-26T06:27:26.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 msgr2=0x7fb9ac19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac19a360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb994000c00 tx=0x7fb99400f690 comp rx=0 tx=0).stop 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 shutdown_connections 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9900a21d0 0x7fb9900a4690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9ac109b80 0x7fb9ac1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9ac074670 0x7fb9ac199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 --2- 172.21.15.6:0/3628488374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9ac075c40 0x7fb9ac19a360 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 >> 172.21.15.6:0/3628488374 conn(0x7fb9ac0fd280 msgr2=0x7fb9ac105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 shutdown_connections 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:26.684+0000 7fb9b2000700 1 -- 172.21.15.6:0/3628488374 wait complete. 2024-02-26T06:27:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:27:27.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:27 smithi006 ceph-mon[31790]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:27.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:27 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3628488374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T06:27:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:27 smithi167 ceph-mon[36229]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:27 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3628488374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T06:27:27.587 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-26T06:27:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:27 smithi090 ceph-mon[35681]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:27 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3628488374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T06:27:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:29.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:29 smithi006 ceph-mon[31790]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:29.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:29 smithi090 ceph-mon[35681]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:29 smithi167 ceph-mon[36229]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:30.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 -- 172.21.15.6:0/1977540730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc100ba0 msgr2=0x7fb9dc100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1977540730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc100fc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0099c0 tx=0x7fb9cc009cd0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 -- 172.21.15.6:0/1977540730 shutdown_connections 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1977540730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc102740 0x7fb9dc108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1977540730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc100fc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1977540730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc101da0 0x7fb9dc102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 -- 172.21.15.6:0/1977540730 >> 172.21.15.6:0/1977540730 conn(0x7fb9dc0fc140 msgr2=0x7fb9dc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 -- 172.21.15.6:0/1977540730 shutdown_connections 2024-02-26T06:27:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.205+0000 7fb9e2400700 1 -- 172.21.15.6:0/1977540730 wait complete. 2024-02-26T06:27:30.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.206+0000 7fb9e2400700 1 Processor -- start 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.206+0000 7fb9e2400700 1 -- start start 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9e2400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9e2400700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc101da0 0x7fb9dc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9e2400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc102740 0x7fb9dc1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9e2400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc1a1a70 con 0x7fb9dc102740 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9e2400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc1a1bb0 con 0x7fb9dc100ba0 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9e2400700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc1a1d40 con 0x7fb9dc101da0 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9dbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc102740 0x7fb9dc1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9d3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc101da0 0x7fb9dc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:59820/0 (socket says 172.21.15.6:59820) 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9db400700 1 -- 172.21.15.6:0/1852792568 learned_addr learned my addr 172.21.15.6:0/1852792568 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.207+0000 7fb9d3e00700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc101da0 0x7fb9dc19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:48616/0 (socket says 172.21.15.6:48616) 2024-02-26T06:27:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9db400700 1 -- 172.21.15.6:0/1852792568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc101da0 msgr2=0x7fb9dc19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9db400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc101da0 0x7fb9dc19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9db400700 1 -- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc102740 msgr2=0x7fb9dc1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9db400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc102740 0x7fb9dc1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9db400700 1 -- 172.21.15.6:0/1852792568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0095f0 con 0x7fb9dc100ba0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9db400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc199f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0098e0 tx=0x7fb9cc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc01d070 con 0x7fb9dc100ba0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc1a1fc0 con 0x7fb9dc100ba0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9cc022470 con 0x7fb9dc100ba0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc017650 con 0x7fb9dc100ba0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.208+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc1a2370 con 0x7fb9dc100ba0 2024-02-26T06:27:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.209+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9dc068960 con 0x7fb9dc100ba0 2024-02-26T06:27:30.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.211+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb9cc0225e0 con 0x7fb9dc100ba0 2024-02-26T06:27:30.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.213+0000 7fb9d8c00700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9c00a21d0 0x7fb9c00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:30.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.213+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb9cc0ceb10 con 0x7fb9dc100ba0 2024-02-26T06:27:30.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.213+0000 7fb9d3e00700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9c00a21d0 0x7fb9c00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:30.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.214+0000 7fb9d3e00700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9c00a21d0 0x7fb9c00a4690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009c60 tx=0x7fb9c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:30.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.215+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9cc05dd20 con 0x7fb9dc100ba0 2024-02-26T06:27:30.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.446+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb9dc10c7d0 con 0x7fb9dc100ba0 2024-02-26T06:27:30.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.448+0000 7fb9d8c00700 1 -- 172.21.15.6:0/1852792568 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5272 (secure 0 0 0) 0x7fb9cc097b00 con 0x7fb9dc100ba0 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 87 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:fsid 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-02-26T06:16:02.582083+0000 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-02-26T06:26:13.838432+0000 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:crush_version 26 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:full_ratio 0.95 2024-02-26T06:27:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:backfillfull_ratio 0.9 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:nearfull_ratio 0.85 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:require_min_compat_client luminous 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:min_compat_client luminous 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:require_osd_release squid 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:stretch_mode_enabled false 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:max_osd 12 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6802/2652019447,v1:172.21.15.6:6803/2652019447] [v2:172.21.15.6:6804/2652019447,v1:172.21.15.6:6805/2652019447] exists,up 4d33ce97-4574-4b39-a887-734262e13d40 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6800/2679347906,v1:172.21.15.90:6801/2679347906] [v2:172.21.15.90:6802/2679347906,v1:172.21.15.90:6803/2679347906] exists,up 71372d37-1d3f-46c1-abc4-3a33d0daf773 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/1414245392,v1:172.21.15.167:6801/1414245392] [v2:172.21.15.167:6802/1414245392,v1:172.21.15.167:6803/1414245392] exists,up f8b7b566-c20f-4c25-83c7-3cc834dcbaca 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6810/3201157546,v1:172.21.15.6:6811/3201157546] [v2:172.21.15.6:6812/3201157546,v1:172.21.15.6:6813/3201157546] exists,up 513bca2a-d4e4-4934-8826-3b204caa92b6 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6808/108774388,v1:172.21.15.90:6809/108774388] [v2:172.21.15.90:6810/108774388,v1:172.21.15.90:6811/108774388] exists,up e2ca8afc-575f-4cdd-8310-2c62fba98c84 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/3058964685,v1:172.21.15.167:6809/3058964685] [v2:172.21.15.167:6810/3058964685,v1:172.21.15.167:6811/3058964685] exists,up d5e79bb3-2052-4a0d-8b43-d6bf454109b8 2024-02-26T06:27:30.451 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6818/643219798,v1:172.21.15.6:6819/643219798] [v2:172.21.15.6:6820/643219798,v1:172.21.15.6:6821/643219798] exists,up 2f9e1699-08b0-422a-9875-5a840fa6c47a 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6816/2240633136,v1:172.21.15.90:6817/2240633136] [v2:172.21.15.90:6818/2240633136,v1:172.21.15.90:6819/2240633136] exists,up 3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/734768771,v1:172.21.15.167:6817/734768771] [v2:172.21.15.167:6818/734768771,v1:172.21.15.167:6819/734768771] exists,up 863e848b-67fd-4bc7-82c5-b3b69df061b9 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6826/1514059134,v1:172.21.15.6:6827/1514059134] [v2:172.21.15.6:6828/1514059134,v1:172.21.15.6:6829/1514059134] exists,up 6ee193e3-93b3-47fb-84b4-39a14c4a060f 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6824/1676953993,v1:172.21.15.90:6825/1676953993] [v2:172.21.15.90:6826/1676953993,v1:172.21.15.90:6827/1676953993] exists,up d0f22982-ab74-4fe3-b273-a2d992a543e5 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/1697184190,v1:172.21.15.167:6825/1697184190] [v2:172.21.15.167:6826/1697184190,v1:172.21.15.167:6827/1697184190] exists,up f631c7a6-a02d-434a-9f64-3b60310f0222 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.e [2,1] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.f [8,11] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.1c [8,11] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.1d [2,4] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.5 [3,9] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.11 [5,9] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.14 [4,6] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.17 [4,8] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.1d [5,9,4,7] 2024-02-26T06:27:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/4032437212 expires 2024-02-27T06:17:11.959803+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2377177802 expires 2024-02-27T06:17:11.959803+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/1027725899 expires 2024-02-27T06:16:36.555104+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3180746931 expires 2024-02-27T06:17:11.959803+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2705487638 expires 2024-02-27T06:16:36.555104+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/483778690 expires 2024-02-27T06:16:36.555104+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/4032437212 expires 2024-02-27T06:17:11.959803+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/1027725899 expires 2024-02-27T06:16:36.555104+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2587275670 expires 2024-02-27T06:16:36.555104+0000 2024-02-26T06:27:30.453 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2348330080 expires 2024-02-27T06:17:11.959803+0000 2024-02-26T06:27:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9c00a21d0 msgr2=0x7fb9c00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9c00a21d0 0x7fb9c00a4690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009c60 tx=0x7fb9c4009260 comp rx=0 tx=0).stop 2024-02-26T06:27:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 msgr2=0x7fb9dc199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc199f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0098e0 tx=0x7fb9cc00bfd0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 shutdown_connections 2024-02-26T06:27:30.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb9c00a21d0 0x7fb9c00a4690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9dc102740 0x7fb9dc1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9dc100ba0 0x7fb9dc199f30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 --2- 172.21.15.6:0/1852792568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc101da0 0x7fb9dc19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.452+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 >> 172.21.15.6:0/1852792568 conn(0x7fb9dc0fc140 msgr2=0x7fb9dc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.453+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 shutdown_connections 2024-02-26T06:27:30.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:30.453+0000 7fb9e2400700 1 -- 172.21.15.6:0/1852792568 wait complete. 2024-02-26T06:27:31.267 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-26T06:27:31.278 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 -- 172.21.15.6:0/1620601201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 msgr2=0x7f113410b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 --2- 172.21.15.6:0/1620601201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f113410b6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f112c009ca0 tx=0x7f112c02c100 comp rx=0 tx=0).stop 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 -- 172.21.15.6:0/1620601201 shutdown_connections 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 --2- 172.21.15.6:0/1620601201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f113410b6e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 --2- 172.21.15.6:0/1620601201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 0x7f1134104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 --2- 172.21.15.6:0/1620601201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f11341036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 -- 172.21.15.6:0/1620601201 >> 172.21.15.6:0/1620601201 conn(0x7f11340fe830 msgr2=0x7f1134100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:31.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 -- 172.21.15.6:0/1620601201 shutdown_connections 2024-02-26T06:27:31.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.517+0000 7f113a200640 1 -- 172.21.15.6:0/1620601201 wait complete. 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.518+0000 7f113a200640 1 Processor -- start 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.518+0000 7f113a200640 1 -- start start 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f113a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f1134199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f113a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 0x7f11341997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f113a200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f11341a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f113a200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a0d80 con 0x7f1134104e50 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f113a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a0ef0 con 0x7f1134104490 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f113a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a10b0 con 0x7f11341032a0 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1139800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f11341a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1138e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f1134199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1138e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f1134199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:48626/0 (socket says 172.21.15.6:48626) 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1139800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f11341a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34684/0 (socket says 172.21.15.6:34684) 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1138e00640 1 -- 172.21.15.6:0/2785679221 learned_addr learned my addr 172.21.15.6:0/2785679221 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1133e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 0x7f11341997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1133e00640 1 -- 172.21.15.6:0/2785679221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 msgr2=0x7f1134199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1133e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f1134199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1133e00640 1 -- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 msgr2=0x7f11341a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1133e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f11341a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1133e00640 1 -- 172.21.15.6:0/2785679221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c009900 con 0x7f1134104490 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.519+0000 7f1138e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f1134199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.520+0000 7f1133e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 0x7f11341997d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f112400e9f0 tx=0x7f112400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.520+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400ccc0 con 0x7f1134104490 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.520+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f112400ce20 con 0x7f1134104490 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.520+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11341a1390 con 0x7f1134104490 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.520+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124018920 con 0x7f1134104490 2024-02-26T06:27:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.520+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11341a18a0 con 0x7f1134104490 2024-02-26T06:27:31.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.521+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113410a560 con 0x7f1134104490 2024-02-26T06:27:31.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.523+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1124018a80 con 0x7f1134104490 2024-02-26T06:27:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.525+0000 7f1131600640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f111c0a20b0 0x7f111c0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.525+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1124014070 con 0x7f1134104490 2024-02-26T06:27:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.525+0000 7f1138e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f111c0a20b0 0x7f111c0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.526+0000 7f1138e00640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f111c0a20b0 0x7f111c0a4570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11280104e0 tx=0x7f112800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:31.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.527+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1124097030 con 0x7f1134104490 2024-02-26T06:27:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:31 smithi090 ceph-mon[35681]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:31 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1852792568' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T06:27:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:31 smithi006 ceph-mon[31790]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:31.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:31 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1852792568' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T06:27:31.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.726+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f113410eef0 con 0x7f1134104490 2024-02-26T06:27:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.729+0000 7f1131600640 1 -- 172.21.15.6:0/2785679221 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f112409a6f0 con 0x7f1134104490 2024-02-26T06:27:31.733 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:31.734 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.731+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f111c0a20b0 msgr2=0x7f111c0a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.731+0000 7f113a200640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f111c0a20b0 0x7f111c0a4570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11280104e0 tx=0x7f112800b410 comp rx=0 tx=0).stop 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.731+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 msgr2=0x7f11341997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.731+0000 7f113a200640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 0x7f11341997d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f112400e9f0 tx=0x7f112400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 shutdown_connections 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1134104e50 0x7f11341a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f111c0a20b0 0x7f111c0a4570 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104490 0x7f11341997d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 --2- 172.21.15.6:0/2785679221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11341032a0 0x7f1134199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 >> 172.21.15.6:0/2785679221 conn(0x7f11340fe830 msgr2=0x7f1134107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 shutdown_connections 2024-02-26T06:27:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.732+0000 7f113a200640 1 -- 172.21.15.6:0/2785679221 wait complete. 2024-02-26T06:27:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:31 smithi167 ceph-mon[36229]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:31 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1852792568' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T06:27:31.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:27:31.756 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 -- 172.21.15.6:0/2374758192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340104d90 msgr2=0x7f934010b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 --2- 172.21.15.6:0/2374758192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340104d90 0x7f934010b620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9338009900 tx=0x7f933802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 -- 172.21.15.6:0/2374758192 shutdown_connections 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 --2- 172.21.15.6:0/2374758192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340104d90 0x7f934010b620 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 --2- 172.21.15.6:0/2374758192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401043d0 0x7f9340104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 --2- 172.21.15.6:0/2374758192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93401031e0 0x7f9340103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 -- 172.21.15.6:0/2374758192 >> 172.21.15.6:0/2374758192 conn(0x7f93400fe750 msgr2=0x7f9340100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.977+0000 7f9346200640 1 -- 172.21.15.6:0/2374758192 shutdown_connections 2024-02-26T06:27:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.978+0000 7f9346200640 1 -- 172.21.15.6:0/2374758192 wait complete. 2024-02-26T06:27:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.978+0000 7f9346200640 1 Processor -- start 2024-02-26T06:27:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 -- start start 2024-02-26T06:27:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 0x7f93401991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401043d0 0x7f9340199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9340104d90 0x7f93401a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0cd0 con 0x7f93401031e0 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0e40 con 0x7f9340104d90 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9346200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a1000 con 0x7f93401043d0 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9344e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 0x7f93401991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9344e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 0x7f93401991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34704/0 (socket says 172.21.15.6:34704) 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9344e00640 1 -- 172.21.15.6:0/3140865057 learned_addr learned my addr 172.21.15.6:0/3140865057 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f933fe00640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401043d0 0x7f9340199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.979+0000 7f9345800640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9340104d90 0x7f93401a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9344e00640 1 -- 172.21.15.6:0/3140865057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401043d0 msgr2=0x7f9340199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9344e00640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401043d0 0x7f9340199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9344e00640 1 -- 172.21.15.6:0/3140865057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9340104d90 msgr2=0x7f93401a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9344e00640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9340104d90 0x7f93401a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9344e00640 1 -- 172.21.15.6:0/3140865057 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330009490 con 0x7f93401031e0 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9345800640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9340104d90 0x7f93401a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9344e00640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 0x7f93401991e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f933000e9c0 tx=0x7f933000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933000cde0 con 0x7f93401031e0 2024-02-26T06:27:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9330010910 con 0x7f93401031e0 2024-02-26T06:27:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330018a10 con 0x7f93401031e0 2024-02-26T06:27:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.980+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9338009560 con 0x7f93401031e0 2024-02-26T06:27:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.981+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93401a1660 con 0x7f93401031e0 2024-02-26T06:27:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.982+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934010a500 con 0x7f93401031e0 2024-02-26T06:27:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.984+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9330010a80 con 0x7f93401031e0 2024-02-26T06:27:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.985+0000 7f933d600640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93240a20c0 0x7f93240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.985+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9330014070 con 0x7f93401031e0 2024-02-26T06:27:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.985+0000 7f933fe00640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93240a20c0 0x7f93240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:31.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.986+0000 7f933fe00640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93240a20c0 0x7f93240a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f934019a6f0 tx=0x7f932c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:31.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:31.988+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9330097950 con 0x7f93401031e0 2024-02-26T06:27:32.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.188+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f934002ce40 con 0x7f93401031e0 2024-02-26T06:27:32.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.189+0000 7f933d600640 1 -- 172.21.15.6:0/3140865057 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f933009b010 con 0x7f93401031e0 2024-02-26T06:27:32.189 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:27:32.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93240a20c0 msgr2=0x7f93240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93240a20c0 0x7f93240a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f934019a6f0 tx=0x7f932c009500 comp rx=0 tx=0).stop 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 msgr2=0x7f93401991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 0x7f93401991e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f933000e9c0 tx=0x7f933000ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 shutdown_connections 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f93240a20c0 0x7f93240a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9340104d90 0x7f93401a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401043d0 0x7f9340199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 --2- 172.21.15.6:0/3140865057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401031e0 0x7f93401991e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 >> 172.21.15.6:0/3140865057 conn(0x7f93400fe750 msgr2=0x7f93401076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 shutdown_connections 2024-02-26T06:27:32.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.192+0000 7f9346200640 1 -- 172.21.15.6:0/3140865057 wait complete. 2024-02-26T06:27:32.213 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:27:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 -- 172.21.15.6:0/2811339775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1043b0 msgr2=0x7f20ec104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 --2- 172.21.15.6:0/2811339775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1043b0 0x7f20ec104830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009900 tx=0x7f20d802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 -- 172.21.15.6:0/2811339775 shutdown_connections 2024-02-26T06:27:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 --2- 172.21.15.6:0/2811339775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec104d70 0x7f20ec10b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 --2- 172.21.15.6:0/2811339775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1043b0 0x7f20ec104830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 --2- 172.21.15.6:0/2811339775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20ec1031c0 0x7f20ec1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 -- 172.21.15.6:0/2811339775 >> 172.21.15.6:0/2811339775 conn(0x7f20ec0fe750 msgr2=0x7f20ec100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 -- 172.21.15.6:0/2811339775 shutdown_connections 2024-02-26T06:27:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.445+0000 7f20ebe00640 1 -- 172.21.15.6:0/2811339775 wait complete. 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.446+0000 7f20ebe00640 1 Processor -- start 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 -- start start 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 0x7f20ec199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec1043b0 0x7f20ec1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20ec104d70 0x7f20ec1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a0d70 con 0x7f20ec1031c0 2024-02-26T06:27:32.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a0ee0 con 0x7f20ec104d70 2024-02-26T06:27:32.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ebe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a10a0 con 0x7f20ec1043b0 2024-02-26T06:27:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20eaa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 0x7f20ec199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20eaa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 0x7f20ec199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34712/0 (socket says 172.21.15.6:34712) 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20eaa00640 1 -- 172.21.15.6:0/4101935815 learned_addr learned my addr 172.21.15.6:0/4101935815 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20ea000640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec1043b0 0x7f20ec1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.447+0000 7f20eb400640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20ec104d70 0x7f20ec1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20eaa00640 1 -- 172.21.15.6:0/4101935815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec1043b0 msgr2=0x7f20ec1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20eaa00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec1043b0 0x7f20ec1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20eaa00640 1 -- 172.21.15.6:0/4101935815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20ec104d70 msgr2=0x7f20ec1a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20eaa00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20ec104d70 0x7f20ec1a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20eaa00640 1 -- 172.21.15.6:0/4101935815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc009490 con 0x7f20ec1031c0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20eaa00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 0x7f20ec199280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00e9c0 tx=0x7f20dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00cde0 con 0x7f20ec1031c0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20dc010910 con 0x7f20ec1031c0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d8009560 con 0x7f20ec1031c0 2024-02-26T06:27:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.448+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc018a10 con 0x7f20ec1031c0 2024-02-26T06:27:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.449+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec1a16d0 con 0x7f20ec1031c0 2024-02-26T06:27:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.450+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec10a4e0 con 0x7f20ec1031c0 2024-02-26T06:27:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.452+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f20dc010a80 con 0x7f20ec1031c0 2024-02-26T06:27:32.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.453+0000 7f20e3400640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20d00a20c0 0x7f20d00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.453+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f20dc014070 con 0x7f20ec1031c0 2024-02-26T06:27:32.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.453+0000 7f20ea000640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20d00a20c0 0x7f20d00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.454+0000 7f20ea000640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20d00a20c0 0x7f20d00a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009230 tx=0x7f20d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:32.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.457+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20dc097340 con 0x7f20ec1031c0 2024-02-26T06:27:32.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:32 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2785679221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:32.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:32 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3140865057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:32.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.660+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20ec02ce40 con 0x7f20ec1031c0 2024-02-26T06:27:32.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.661+0000 7f20e3400640 1 -- 172.21.15.6:0/4101935815 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20dc09aa00 con 0x7f20ec1031c0 2024-02-26T06:27:32.661 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:27:32.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20d00a20c0 msgr2=0x7f20d00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20d00a20c0 0x7f20d00a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009230 tx=0x7f20d8005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 msgr2=0x7f20ec199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 0x7f20ec199280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00e9c0 tx=0x7f20dc00ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 shutdown_connections 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f20d00a20c0 0x7f20d00a4580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20ec104d70 0x7f20ec1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec1043b0 0x7f20ec1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 --2- 172.21.15.6:0/4101935815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec1031c0 0x7f20ec199280 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 >> 172.21.15.6:0/4101935815 conn(0x7f20ec0fe750 msgr2=0x7f20ec1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 shutdown_connections 2024-02-26T06:27:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.664+0000 7f20ebe00640 1 -- 172.21.15.6:0/4101935815 wait complete. 2024-02-26T06:27:32.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:32 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2785679221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:32.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:32 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3140865057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:32.684 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:27:32.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:32 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2785679221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:32.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:32 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3140865057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 -- 172.21.15.6:0/1800462742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa581043b0 msgr2=0x7faa58104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 --2- 172.21.15.6:0/1800462742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa581043b0 0x7faa58104830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faa44009900 tx=0x7faa4402bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 -- 172.21.15.6:0/1800462742 shutdown_connections 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 --2- 172.21.15.6:0/1800462742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa58104d70 0x7faa5810b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 --2- 172.21.15.6:0/1800462742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa581043b0 0x7faa58104830 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 --2- 172.21.15.6:0/1800462742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa581031c0 0x7faa581035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 -- 172.21.15.6:0/1800462742 >> 172.21.15.6:0/1800462742 conn(0x7faa580fe750 msgr2=0x7faa58100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.915+0000 7faa5e800640 1 -- 172.21.15.6:0/1800462742 shutdown_connections 2024-02-26T06:27:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.916+0000 7faa5e800640 1 -- 172.21.15.6:0/1800462742 wait complete. 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.916+0000 7faa5e800640 1 Processor -- start 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 -- start start 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 0x7faa58199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa581043b0 0x7faa581997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa58104d70 0x7faa581a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581a0d70 con 0x7faa581031c0 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581a0ee0 con 0x7faa58104d70 2024-02-26T06:27:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581a10a0 con 0x7faa581043b0 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5d400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 0x7faa58199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5d400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 0x7faa58199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34734/0 (socket says 172.21.15.6:34734) 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5d400640 1 -- 172.21.15.6:0/1579326155 learned_addr learned my addr 172.21.15.6:0/1579326155 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.917+0000 7faa5de00640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa58104d70 0x7faa581a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5ca00640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa581043b0 0x7faa581997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5de00640 1 -- 172.21.15.6:0/1579326155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa581043b0 msgr2=0x7faa581997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5de00640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa581043b0 0x7faa581997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5de00640 1 -- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 msgr2=0x7faa58199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5de00640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 0x7faa58199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5de00640 1 -- 172.21.15.6:0/1579326155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44009560 con 0x7faa58104d70 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5d400640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 0x7faa58199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa5de00640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa58104d70 0x7faa581a0830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa5000da10 tx=0x7faa5000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.918+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5000bd70 con 0x7faa58104d70 2024-02-26T06:27:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.919+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa50003e50 con 0x7faa58104d70 2024-02-26T06:27:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.919+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa500106c0 con 0x7faa58104d70 2024-02-26T06:27:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.919+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa581a1380 con 0x7faa58104d70 2024-02-26T06:27:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.919+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa581a1810 con 0x7faa58104d70 2024-02-26T06:27:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.920+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5810a5d0 con 0x7faa58104d70 2024-02-26T06:27:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.922+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7faa50004000 con 0x7faa58104d70 2024-02-26T06:27:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.923+0000 7faa56000640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faa400a20c0 0x7faa400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.923+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7faa500cdfc0 con 0x7faa58104d70 2024-02-26T06:27:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.924+0000 7faa5d400640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faa400a20c0 0x7faa400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.925+0000 7faa5d400640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faa400a20c0 0x7faa400a4580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0104e0 tx=0x7faa4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:32.926+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa50096f80 con 0x7faa58104d70 2024-02-26T06:27:33.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.128+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa5802ce40 con 0x7faa58104d70 2024-02-26T06:27:33.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.129+0000 7faa56000640 1 -- 172.21.15.6:0/1579326155 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa5009a640 con 0x7faa58104d70 2024-02-26T06:27:33.130 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:27:33.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.131+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faa400a20c0 msgr2=0x7faa400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.131+0000 7faa5e800640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faa400a20c0 0x7faa400a4580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0104e0 tx=0x7faa4c00b410 comp rx=0 tx=0).stop 2024-02-26T06:27:33.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.131+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa58104d70 msgr2=0x7faa581a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.131+0000 7faa5e800640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa58104d70 0x7faa581a0830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa5000da10 tx=0x7faa5000ded0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 shutdown_connections 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7faa400a20c0 0x7faa400a4580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa58104d70 0x7faa581a0830 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa581043b0 0x7faa581997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 --2- 172.21.15.6:0/1579326155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa581031c0 0x7faa58199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 >> 172.21.15.6:0/1579326155 conn(0x7faa580fe750 msgr2=0x7faa581077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 shutdown_connections 2024-02-26T06:27:33.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.132+0000 7faa5e800640 1 -- 172.21.15.6:0/1579326155 wait complete. 2024-02-26T06:27:33.153 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 -- 172.21.15.6:0/3836877828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 msgr2=0x7f16900735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 --2- 172.21.15.6:0/3836877828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f16900735b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1684009900 tx=0x7f168402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 -- 172.21.15.6:0/3836877828 shutdown_connections 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 --2- 172.21.15.6:0/3836877828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 0x7f169010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 --2- 172.21.15.6:0/3836877828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1690073b80 0x7f1690073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 --2- 172.21.15.6:0/3836877828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f16900735b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 -- 172.21.15.6:0/3836877828 >> 172.21.15.6:0/3836877828 conn(0x7f16900fc610 msgr2=0x7f16900fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:33.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 -- 172.21.15.6:0/3836877828 shutdown_connections 2024-02-26T06:27:33.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.374+0000 7f1697200640 1 -- 172.21.15.6:0/3836877828 wait complete. 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.375+0000 7f1697200640 1 Processor -- start 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.375+0000 7f1697200640 1 -- start start 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1697200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1690073b80 0x7f1690199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1697200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f1690199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1697200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 0x7f16901a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1697200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901a0e10 con 0x7f1690108f30 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1697200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901a0f80 con 0x7f1690075160 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1697200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901a1140 con 0x7f1690073b80 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1695400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f1690199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1696800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 0x7f16901a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1695400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f1690199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:59930/0 (socket says 172.21.15.6:59930) 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1695400640 1 -- 172.21.15.6:0/683667167 learned_addr learned my addr 172.21.15.6:0/683667167 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1695e00640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1690073b80 0x7f1690199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1696800640 1 -- 172.21.15.6:0/683667167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1690073b80 msgr2=0x7f1690199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1696800640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1690073b80 0x7f1690199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1696800640 1 -- 172.21.15.6:0/683667167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 msgr2=0x7f1690199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.376+0000 7f1696800640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f1690199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f1696800640 1 -- 172.21.15.6:0/683667167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680009490 con 0x7f1690108f30 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f1695400640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f1690199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f1696800640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 0x7f16901a08d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f168c00ec50 tx=0x7f168c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c003b70 con 0x7f1690108f30 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c009840 con 0x7f1690108f30 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f168c00ce70 con 0x7f1690108f30 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c010830 con 0x7f1690108f30 2024-02-26T06:27:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.377+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684009560 con 0x7f1690108f30 2024-02-26T06:27:33.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.378+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1690107cf0 con 0x7f1690108f30 2024-02-26T06:27:33.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.380+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f168c004020 con 0x7f1690108f30 2024-02-26T06:27:33.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.382+0000 7f168aa00640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f16780a20b0 0x7f16780a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.382+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f168c014070 con 0x7f1690108f30 2024-02-26T06:27:33.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.382+0000 7f1695e00640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f16780a20b0 0x7f16780a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.383+0000 7f1695e00640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f16780a20b0 0x7f16780a4570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1684009230 tx=0x7f1684005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:33.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.386+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f168c097380 con 0x7f1690108f30 2024-02-26T06:27:33.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:33 smithi006 ceph-mon[31790]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:33 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4101935815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:33.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:33 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1579326155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:33 smithi090 ceph-mon[35681]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:33 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4101935815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:33 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1579326155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:33.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.627+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f169002cda0 con 0x7f1690108f30 2024-02-26T06:27:33.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.629+0000 7f168aa00640 1 -- 172.21.15.6:0/683667167 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f168c09aa40 con 0x7f1690108f30 2024-02-26T06:27:33.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:33.629 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:25:24.777181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:33.630 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:27:33.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.631+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f16780a20b0 msgr2=0x7f16780a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.631+0000 7f1697200640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f16780a20b0 0x7f16780a4570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1684009230 tx=0x7f1684005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.631+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 msgr2=0x7f16901a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.631+0000 7f1697200640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 0x7f16901a08d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f168c00ec50 tx=0x7f168c00c6a0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.631+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 shutdown_connections 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.631+0000 7f1697200640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1690108f30 0x7f16901a08d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.632+0000 7f1697200640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f16780a20b0 0x7f16780a4570 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.632+0000 7f1697200640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1690075160 0x7f1690199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.632+0000 7f1697200640 1 --2- 172.21.15.6:0/683667167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1690073b80 0x7f1690199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.632+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 >> 172.21.15.6:0/683667167 conn(0x7f16900fc610 msgr2=0x7f1690104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.632+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 shutdown_connections 2024-02-26T06:27:33.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.632+0000 7f1697200640 1 -- 172.21.15.6:0/683667167 wait complete. 2024-02-26T06:27:33.651 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:33 smithi167 ceph-mon[36229]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:33 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4101935815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:33 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1579326155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 -- 172.21.15.6:0/3924475504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 msgr2=0x7fe76c10b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 --2- 172.21.15.6:0/3924475504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c10b6e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe764009be0 tx=0x7fe76402bef0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 -- 172.21.15.6:0/3924475504 shutdown_connections 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 --2- 172.21.15.6:0/3924475504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c10b6e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 --2- 172.21.15.6:0/3924475504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c104490 0x7fe76c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 --2- 172.21.15.6:0/3924475504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c1032a0 0x7fe76c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 -- 172.21.15.6:0/3924475504 >> 172.21.15.6:0/3924475504 conn(0x7fe76c0fe830 msgr2=0x7fe76c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.875+0000 7fe772800640 1 -- 172.21.15.6:0/3924475504 shutdown_connections 2024-02-26T06:27:33.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.876+0000 7fe772800640 1 -- 172.21.15.6:0/3924475504 wait complete. 2024-02-26T06:27:33.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.876+0000 7fe772800640 1 Processor -- start 2024-02-26T06:27:33.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 -- start start 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 0x7fe76c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c104490 0x7fe76c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1a0df0 con 0x7fe76c1032a0 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1a0f60 con 0x7fe76c104e50 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe772800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1a1120 con 0x7fe76c104490 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe771400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 0x7fe76c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe771e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe770a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c104490 0x7fe76c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe771400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 0x7fe76c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34770/0 (socket says 172.21.15.6:34770) 2024-02-26T06:27:33.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe771e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:59948/0 (socket says 172.21.15.6:59948) 2024-02-26T06:27:33.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.877+0000 7fe771400640 1 -- 172.21.15.6:0/2219498781 learned_addr learned my addr 172.21.15.6:0/2219498781 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771400640 1 -- 172.21.15.6:0/2219498781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c104490 msgr2=0x7fe76c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771400640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c104490 0x7fe76c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771400640 1 -- 172.21.15.6:0/2219498781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 msgr2=0x7fe76c1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771400640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771400640 1 -- 172.21.15.6:0/2219498781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764009840 con 0x7fe76c1032a0 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe770a00640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c104490 0x7fe76c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771e00640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.878+0000 7fe771400640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 0x7fe76c199300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e9f0 tx=0x7fe76000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.879+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000ccc0 con 0x7fe76c1032a0 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.879+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c1a1400 con 0x7fe76c1032a0 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.879+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe76000ce20 con 0x7fe76c1032a0 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.879+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760010430 con 0x7fe76c1032a0 2024-02-26T06:27:33.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.879+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c1a1960 con 0x7fe76c1032a0 2024-02-26T06:27:33.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.880+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c10a560 con 0x7fe76c1032a0 2024-02-26T06:27:33.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.882+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe760003fb0 con 0x7fe76c1032a0 2024-02-26T06:27:33.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.884+0000 7fe76a000640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe7540a20b0 0x7fe7540a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:33.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.884+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe760014070 con 0x7fe76c1032a0 2024-02-26T06:27:33.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.884+0000 7fe770a00640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe7540a20b0 0x7fe7540a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:33.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.885+0000 7fe770a00640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe7540a20b0 0x7fe7540a4570 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe76c19a810 tx=0x7fe758005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:33.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:33.889+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe760096ee0 con 0x7fe76c1032a0 2024-02-26T06:27:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.091+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe76c02cda0 con 0x7fe76c1032a0 2024-02-26T06:27:34.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.093+0000 7fe76a000640 1 -- 172.21.15.6:0/2219498781 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7fe76009a5a0 con 0x7fe76c1032a0 2024-02-26T06:27:34.093 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:34.094 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe7540a20b0 msgr2=0x7fe7540a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe7540a20b0 0x7fe7540a4570 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe76c19a810 tx=0x7fe758005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 msgr2=0x7fe76c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 0x7fe76c199300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e9f0 tx=0x7fe76000eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 shutdown_connections 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe7540a20b0 0x7fe7540a4570 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe76c104e50 0x7fe76c1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe76c104490 0x7fe76c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 --2- 172.21.15.6:0/2219498781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe76c1032a0 0x7fe76c199300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 >> 172.21.15.6:0/2219498781 conn(0x7fe76c0fe830 msgr2=0x7fe76c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 shutdown_connections 2024-02-26T06:27:34.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.096+0000 7fe772800640 1 -- 172.21.15.6:0/2219498781 wait complete. 2024-02-26T06:27:34.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:27:34.115 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:27:34.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 -- 172.21.15.6:0/2558021036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c1033b0 msgr2=0x7f9b8c1037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 --2- 172.21.15.6:0/2558021036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1037d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009ca0 tx=0x7f9b7c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 -- 172.21.15.6:0/2558021036 shutdown_connections 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 --2- 172.21.15.6:0/2558021036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c104f60 0x7f9b8c10b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 --2- 172.21.15.6:0/2558021036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8c1045a0 0x7f9b8c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 --2- 172.21.15.6:0/2558021036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1037d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 -- 172.21.15.6:0/2558021036 >> 172.21.15.6:0/2558021036 conn(0x7f9b8c0fe940 msgr2=0x7f9b8c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 -- 172.21.15.6:0/2558021036 shutdown_connections 2024-02-26T06:27:34.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.337+0000 7f9b92a00640 1 -- 172.21.15.6:0/2558021036 wait complete. 2024-02-26T06:27:34.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.338+0000 7f9b92a00640 1 Processor -- start 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.338+0000 7f9b92a00640 1 -- start start 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b92a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8c1045a0 0x7f9b8c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b92a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 0x7f9b8c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b92a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1a0fb0 con 0x7f9b8c104f60 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b92a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1a1120 con 0x7f9b8c1045a0 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b92a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1a12e0 con 0x7f9b8c1033b0 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b90a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 0x7f9b8c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b90a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 0x7f9b8c1a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34800/0 (socket says 172.21.15.6:34800) 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b90a00640 1 -- 172.21.15.6:0/1764451433 learned_addr learned my addr 172.21.15.6:0/1764451433 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.339+0000 7f9b83400640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8c1045a0 0x7f9b8c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b90a00640 1 -- 172.21.15.6:0/1764451433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c1033b0 msgr2=0x7f9b8c1994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b90a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b90a00640 1 -- 172.21.15.6:0/1764451433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8c1045a0 msgr2=0x7f9b8c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b90a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8c1045a0 0x7f9b8c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b90a00640 1 -- 172.21.15.6:0/1764451433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c009900 con 0x7f9b8c104f60 2024-02-26T06:27:34.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b8be00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b90a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 0x7f9b8c1a0a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400d950 tx=0x7f9b8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84009a30 con 0x7f9b8c104f60 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b84009b90 con 0x7f9b8c104f60 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c1a15c0 con 0x7f9b8c104f60 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84003d60 con 0x7f9b8c104f60 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.340+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c1a1a80 con 0x7f9b8c104f60 2024-02-26T06:27:34.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.341+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c10a770 con 0x7f9b8c104f60 2024-02-26T06:27:34.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.343+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9b84009d00 con 0x7f9b8c104f60 2024-02-26T06:27:34.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.345+0000 7f9b89600640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9b700a20c0 0x7f9b700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.345+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9b840cd6c0 con 0x7f9b8c104f60 2024-02-26T06:27:34.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.345+0000 7f9b8be00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9b700a20c0 0x7f9b700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.346+0000 7f9b8be00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9b700a20c0 0x7f9b700a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c02c630 tx=0x7f9b7c009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:34.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.349+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b84096620 con 0x7f9b8c104f60 2024-02-26T06:27:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:34 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/683667167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:34 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2219498781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.549+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b8c02cdd0 con 0x7f9b8c104f60 2024-02-26T06:27:34.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.550+0000 7f9b89600640 1 -- 172.21.15.6:0/1764451433 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b84099ce0 con 0x7f9b8c104f60 2024-02-26T06:27:34.550 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.552+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9b700a20c0 msgr2=0x7f9b700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.552+0000 7f9b92a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9b700a20c0 0x7f9b700a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c02c630 tx=0x7f9b7c009800 comp rx=0 tx=0).stop 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.552+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 msgr2=0x7f9b8c1a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.552+0000 7f9b92a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 0x7f9b8c1a0a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400d950 tx=0x7f9b8400de10 comp rx=0 tx=0).stop 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 shutdown_connections 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b8c104f60 0x7f9b8c1a0a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f9b700a20c0 0x7f9b700a4580 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8c1045a0 0x7f9b8c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 --2- 172.21.15.6:0/1764451433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c1033b0 0x7f9b8c1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 >> 172.21.15.6:0/1764451433 conn(0x7f9b8c0fe940 msgr2=0x7f9b8c107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 shutdown_connections 2024-02-26T06:27:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.553+0000 7f9b92a00640 1 -- 172.21.15.6:0/1764451433 wait complete. 2024-02-26T06:27:34.574 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:27:34.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:34 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/683667167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:34.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:34 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2219498781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:34.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:34 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/683667167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:34.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:34 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2219498781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:34.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 -- 172.21.15.6:0/743786854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e4104d70 msgr2=0x7f87e410b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 --2- 172.21.15.6:0/743786854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e4104d70 0x7f87e410b600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009ca0 tx=0x7f87dc02c100 comp rx=0 tx=0).stop 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 -- 172.21.15.6:0/743786854 shutdown_connections 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 --2- 172.21.15.6:0/743786854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e4104d70 0x7f87e410b600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 --2- 172.21.15.6:0/743786854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87e41043b0 0x7f87e4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 --2- 172.21.15.6:0/743786854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41031c0 0x7f87e41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 -- 172.21.15.6:0/743786854 >> 172.21.15.6:0/743786854 conn(0x7f87e40fe750 msgr2=0x7f87e4100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 -- 172.21.15.6:0/743786854 shutdown_connections 2024-02-26T06:27:34.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.795+0000 7f87e9c00640 1 -- 172.21.15.6:0/743786854 wait complete. 2024-02-26T06:27:34.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.796+0000 7f87e9c00640 1 Processor -- start 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 -- start start 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 0x7f87e4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41043b0 0x7f87e4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87e4104d70 0x7f87e41a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41a0d10 con 0x7f87e41031c0 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41a0e80 con 0x7f87e4104d70 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41a1040 con 0x7f87e41043b0 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 0x7f87e4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 0x7f87e4199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34816/0 (socket says 172.21.15.6:34816) 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e3e00640 1 -- 172.21.15.6:0/499214838 learned_addr learned my addr 172.21.15.6:0/499214838 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e9200640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87e4104d70 0x7f87e41a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.797+0000 7f87e3400640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41043b0 0x7f87e4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3e00640 1 -- 172.21.15.6:0/499214838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41043b0 msgr2=0x7f87e4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3e00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41043b0 0x7f87e4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3e00640 1 -- 172.21.15.6:0/499214838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87e4104d70 msgr2=0x7f87e41a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3e00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87e4104d70 0x7f87e41a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3e00640 1 -- 172.21.15.6:0/499214838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009900 con 0x7f87e41031c0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3400640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41043b0 0x7f87e4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e3e00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 0x7f87e4199220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87d400e9c0 tx=0x7f87d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40096e0 con 0x7f87e41031c0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e41a1320 con 0x7f87e41031c0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d400cd80 con 0x7f87e41031c0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41a1830 con 0x7f87e41031c0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.798+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4010430 con 0x7f87e41031c0 2024-02-26T06:27:34.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.800+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e410a4e0 con 0x7f87e41031c0 2024-02-26T06:27:34.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.802+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f87d4003680 con 0x7f87e41031c0 2024-02-26T06:27:34.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.803+0000 7f87e0c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87c80a20c0 0x7f87c80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:34.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.803+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f87d4014070 con 0x7f87e41031c0 2024-02-26T06:27:34.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.803+0000 7f87e3400640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87c80a20c0 0x7f87c80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:34.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.804+0000 7f87e3400640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87c80a20c0 0x7f87c80a4580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87e419a730 tx=0x7f87cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:34.807+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87d4097190 con 0x7f87e41031c0 2024-02-26T06:27:35.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.006+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87e410ebd0 con 0x7f87e41031c0 2024-02-26T06:27:35.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.007+0000 7f87e0c00640 1 -- 172.21.15.6:0/499214838 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87d409a850 con 0x7f87e41031c0 2024-02-26T06:27:35.007 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87c80a20c0 msgr2=0x7f87c80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87c80a20c0 0x7f87c80a4580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87e419a730 tx=0x7f87cc005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 msgr2=0x7f87e4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 0x7f87e4199220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87d400e9c0 tx=0x7f87d400ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 shutdown_connections 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f87c80a20c0 0x7f87c80a4580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87e4104d70 0x7f87e41a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e41043b0 0x7f87e4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 --2- 172.21.15.6:0/499214838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87e41031c0 0x7f87e4199220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.009+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 >> 172.21.15.6:0/499214838 conn(0x7f87e40fe750 msgr2=0x7f87e41076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.010+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 shutdown_connections 2024-02-26T06:27:35.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.010+0000 7f87e9c00640 1 -- 172.21.15.6:0/499214838 wait complete. 2024-02-26T06:27:35.030 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:27:35.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 -- 172.21.15.6:0/2058438955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1501043d0 msgr2=0x7fb150104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 --2- 172.21.15.6:0/2058438955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1501043d0 0x7fb150104850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb138009900 tx=0x7fb13802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:35.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 -- 172.21.15.6:0/2058438955 shutdown_connections 2024-02-26T06:27:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 --2- 172.21.15.6:0/2058438955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb150104d90 0x7fb15010b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 --2- 172.21.15.6:0/2058438955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1501043d0 0x7fb150104850 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 --2- 172.21.15.6:0/2058438955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501031e0 0x7fb150103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 -- 172.21.15.6:0/2058438955 >> 172.21.15.6:0/2058438955 conn(0x7fb1500fe750 msgr2=0x7fb150100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 -- 172.21.15.6:0/2058438955 shutdown_connections 2024-02-26T06:27:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.256+0000 7fb154a00640 1 -- 172.21.15.6:0/2058438955 wait complete. 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.257+0000 7fb154a00640 1 Processor -- start 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.257+0000 7fb154a00640 1 -- start start 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb154a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 0x7fb150199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb154a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501043d0 0x7fb1501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb154a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb150104d90 0x7fb1501a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb154a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501a0d50 con 0x7fb1501043d0 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb154a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501a0ec0 con 0x7fb1501031e0 2024-02-26T06:27:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb154a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501a1080 con 0x7fb150104d90 2024-02-26T06:27:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 0x7fb150199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501043d0 0x7fb1501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 0x7fb150199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60026/0 (socket says 172.21.15.6:60026) 2024-02-26T06:27:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 -- 172.21.15.6:0/3269211531 learned_addr learned my addr 172.21.15.6:0/3269211531 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14f200640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb150104d90 0x7fb1501a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 -- 172.21.15.6:0/3269211531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb150104d90 msgr2=0x7fb1501a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb150104d90 0x7fb1501a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 -- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501043d0 msgr2=0x7fb1501997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.258+0000 7fb14e800640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501043d0 0x7fb1501997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb14e800640 1 -- 172.21.15.6:0/3269211531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140009490 con 0x7fb1501031e0 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb14de00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501043d0 0x7fb1501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb14e800640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 0x7fb150199260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9c0 tx=0x7fb14000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14000cde0 con 0x7fb1501031e0 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb140010910 con 0x7fb1501031e0 2024-02-26T06:27:35.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140018a10 con 0x7fb1501031e0 2024-02-26T06:27:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb138009560 con 0x7fb1501031e0 2024-02-26T06:27:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.259+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1501a1660 con 0x7fb1501031e0 2024-02-26T06:27:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.260+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15010a500 con 0x7fb1501031e0 2024-02-26T06:27:35.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.263+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb140010a80 con 0x7fb1501031e0 2024-02-26T06:27:35.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.264+0000 7fb147400640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb1340a2140 0x7fb1340a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.264+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb140014070 con 0x7fb1501031e0 2024-02-26T06:27:35.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.264+0000 7fb14de00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb1340a2140 0x7fb1340a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.265+0000 7fb14de00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb1340a2140 0x7fb1340a4600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb138009230 tx=0x7fb138005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:35.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.266+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1400978a0 con 0x7fb1501031e0 2024-02-26T06:27:35.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.467+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb15010ee60 con 0x7fb1501031e0 2024-02-26T06:27:35.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.468+0000 7fb147400640 1 -- 172.21.15.6:0/3269211531 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb140097020 con 0x7fb1501031e0 2024-02-26T06:27:35.468 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.470+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb1340a2140 msgr2=0x7fb1340a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.470+0000 7fb154a00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb1340a2140 0x7fb1340a4600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb138009230 tx=0x7fb138005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.470+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 msgr2=0x7fb150199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.470+0000 7fb154a00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 0x7fb150199260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9c0 tx=0x7fb14000ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 shutdown_connections 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb150104d90 0x7fb1501a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1501043d0 0x7fb1501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb1340a2140 0x7fb1340a4600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 --2- 172.21.15.6:0/3269211531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1501031e0 0x7fb150199260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 >> 172.21.15.6:0/3269211531 conn(0x7fb1500fe750 msgr2=0x7fb1501076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:35.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 shutdown_connections 2024-02-26T06:27:35.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.471+0000 7fb154a00640 1 -- 172.21.15.6:0/3269211531 wait complete. 2024-02-26T06:27:35.491 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:35.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:35 smithi006 ceph-mon[31790]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:35.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:35 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1764451433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:35.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:35 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/499214838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:35.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:35 smithi090 ceph-mon[35681]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:35.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:35 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1764451433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:35.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:35 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/499214838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 -- 172.21.15.6:0/1683180952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 msgr2=0x7fde94077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 --2- 172.21.15.6:0/1683180952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde94077fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009ca0 tx=0x7fde8c02c100 comp rx=0 tx=0).stop 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 -- 172.21.15.6:0/1683180952 shutdown_connections 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 --2- 172.21.15.6:0/1683180952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde94077fc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 --2- 172.21.15.6:0/1683180952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94076d80 0x7fde940771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 --2- 172.21.15.6:0/1683180952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde94075b40 0x7fde94075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 -- 172.21.15.6:0/1683180952 >> 172.21.15.6:0/1683180952 conn(0x7fde940fe830 msgr2=0x7fde94100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 -- 172.21.15.6:0/1683180952 shutdown_connections 2024-02-26T06:27:35.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.719+0000 7fde9a600640 1 -- 172.21.15.6:0/1683180952 wait complete. 2024-02-26T06:27:35.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.720+0000 7fde9a600640 1 Processor -- start 2024-02-26T06:27:35.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 -- start start 2024-02-26T06:27:35.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 0x7fde9419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde94076d80 0x7fde9419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde941a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941a5160 con 0x7fde94075b40 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941a52d0 con 0x7fde94077720 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde9a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941a5490 con 0x7fde94076d80 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde99200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 0x7fde9419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde99200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 0x7fde9419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34856/0 (socket says 172.21.15.6:34856) 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde99200640 1 -- 172.21.15.6:0/2227567859 learned_addr learned my addr 172.21.15.6:0/2227567859 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde99c00640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde941a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.721+0000 7fde93e00640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde94076d80 0x7fde9419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99200640 1 -- 172.21.15.6:0/2227567859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde94076d80 msgr2=0x7fde9419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99200640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde94076d80 0x7fde9419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99200640 1 -- 172.21.15.6:0/2227567859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 msgr2=0x7fde941a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99200640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde941a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99200640 1 -- 172.21.15.6:0/2227567859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88009490 con 0x7fde94075b40 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99c00640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde941a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:35.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde99200640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 0x7fde9419d670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde8800ea60 tx=0x7fde8800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:35.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.722+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8800cde0 con 0x7fde94075b40 2024-02-26T06:27:35.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.723+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde880035f0 con 0x7fde94075b40 2024-02-26T06:27:35.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.723+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c009900 con 0x7fde94075b40 2024-02-26T06:27:35.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.723+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88010690 con 0x7fde94075b40 2024-02-26T06:27:35.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.723+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde941a5af0 con 0x7fde94075b40 2024-02-26T06:27:35.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.724+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9410fc10 con 0x7fde94075b40 2024-02-26T06:27:35.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.726+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fde88003af0 con 0x7fde94075b40 2024-02-26T06:27:35.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.728+0000 7fde91600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fde7c0a20b0 0x7fde7c0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:35.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.728+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fde88014070 con 0x7fde94075b40 2024-02-26T06:27:35.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.728+0000 7fde93e00640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fde7c0a20b0 0x7fde7c0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:35.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.729+0000 7fde93e00640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fde7c0a20b0 0x7fde7c0a4570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde9419eb80 tx=0x7fde8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:35.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.732+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde880970e0 con 0x7fde94075b40 2024-02-26T06:27:35.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:35 smithi167 ceph-mon[36229]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:35.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:35 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1764451433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:35.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:35 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/499214838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:35.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.974+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde941130a0 con 0x7fde94075b40 2024-02-26T06:27:35.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.975+0000 7fde91600640 1 -- 172.21.15.6:0/2227567859 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11356 (secure 0 0 0) 0x7fde8809a7a0 con 0x7fde94075b40 2024-02-26T06:27:35.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:35.976 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:25:24.777181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:35.976 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2024-02-26T06:27:35.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fde7c0a20b0 msgr2=0x7fde7c0a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fde7c0a20b0 0x7fde7c0a4570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde9419eb80 tx=0x7fde8000b410 comp rx=0 tx=0).stop 2024-02-26T06:27:35.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 msgr2=0x7fde9419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:35.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 0x7fde9419d670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde8800ea60 tx=0x7fde8800ef20 comp rx=0 tx=0).stop 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 shutdown_connections 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fde7c0a20b0 0x7fde7c0a4570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde94077720 0x7fde941a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde94076d80 0x7fde9419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 --2- 172.21.15.6:0/2227567859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde94075b40 0x7fde9419d670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.978+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 >> 172.21.15.6:0/2227567859 conn(0x7fde940fe830 msgr2=0x7fde94100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.979+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 shutdown_connections 2024-02-26T06:27:35.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:35.979+0000 7fde9a600640 1 -- 172.21.15.6:0/2227567859 wait complete. 2024-02-26T06:27:35.997 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:36.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 -- 172.21.15.6:0/3531505177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1981076a0 msgr2=0x7fd198107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 --2- 172.21.15.6:0/3531505177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1981076a0 0x7fd198107ac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd188009ca0 tx=0x7fd18802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 -- 172.21.15.6:0/3531505177 shutdown_connections 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 --2- 172.21.15.6:0/3531505177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd198109250 0x7fd19810fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 --2- 172.21.15.6:0/3531505177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198108890 0x7fd198108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 --2- 172.21.15.6:0/3531505177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd1981076a0 0x7fd198107ac0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 -- 172.21.15.6:0/3531505177 >> 172.21.15.6:0/3531505177 conn(0x7fd198076640 msgr2=0x7fd198078a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 -- 172.21.15.6:0/3531505177 shutdown_connections 2024-02-26T06:27:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.221+0000 7fd19d600640 1 -- 172.21.15.6:0/3531505177 wait complete. 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 Processor -- start 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 -- start start 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1981076a0 0x7fd19819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd198108890 0x7fd19819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd198109250 0x7fd1981a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a51d0 con 0x7fd198109250 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a5340 con 0x7fd1981076a0 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.222+0000 7fd19d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a5500 con 0x7fd198108890 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd197e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1981076a0 0x7fd19819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd19cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd198109250 0x7fd1981a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd197e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1981076a0 0x7fd19819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60062/0 (socket says 172.21.15.6:60062) 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd197e00640 1 -- 172.21.15.6:0/3793386010 learned_addr learned my addr 172.21.15.6:0/3793386010 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd197400640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd198108890 0x7fd19819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd19cc00640 1 -- 172.21.15.6:0/3793386010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd198108890 msgr2=0x7fd19819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd19cc00640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd198108890 0x7fd19819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd19cc00640 1 -- 172.21.15.6:0/3793386010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1981076a0 msgr2=0x7fd19819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd19cc00640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1981076a0 0x7fd19819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.223+0000 7fd19cc00640 1 -- 172.21.15.6:0/3793386010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184009490 con 0x7fd198109250 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.224+0000 7fd19cc00640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd198109250 0x7fd1981a4c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e970 tx=0x7fd19000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.224+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190009620 con 0x7fd198109250 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.224+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd19000ccc0 con 0x7fd198109250 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.224+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190004a00 con 0x7fd198109250 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.224+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188009900 con 0x7fd198109250 2024-02-26T06:27:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.224+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1981a5b10 con 0x7fd198109250 2024-02-26T06:27:36.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.225+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19810e9c0 con 0x7fd198109250 2024-02-26T06:27:36.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.227+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd190005020 con 0x7fd198109250 2024-02-26T06:27:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.228+0000 7fd194c00640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd17c0a2130 0x7fd17c0a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.228+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd190014070 con 0x7fd198109250 2024-02-26T06:27:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.228+0000 7fd197e00640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd17c0a2130 0x7fd17c0a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.229+0000 7fd197e00640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd17c0a2130 0x7fd17c0a45f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd188005e30 tx=0x7fd188004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.232+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd190096b40 con 0x7fd198109250 2024-02-26T06:27:36.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.427+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd19802ce40 con 0x7fd198109250 2024-02-26T06:27:36.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.429+0000 7fd194c00640 1 -- 172.21.15.6:0/3793386010 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7fd19009a200 con 0x7fd198109250 2024-02-26T06:27:36.430 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:36.430 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.431+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd17c0a2130 msgr2=0x7fd17c0a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.431+0000 7fd19d600640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd17c0a2130 0x7fd17c0a45f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd188005e30 tx=0x7fd188004a40 comp rx=0 tx=0).stop 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.431+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd198109250 msgr2=0x7fd1981a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.431+0000 7fd19d600640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd198109250 0x7fd1981a4c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e970 tx=0x7fd19000ee30 comp rx=0 tx=0).stop 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 shutdown_connections 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd198109250 0x7fd1981a4c90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd198108890 0x7fd19819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd17c0a2130 0x7fd17c0a45f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 --2- 172.21.15.6:0/3793386010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1981076a0 0x7fd19819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 >> 172.21.15.6:0/3793386010 conn(0x7fd198076640 msgr2=0x7fd19810bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 shutdown_connections 2024-02-26T06:27:36.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.432+0000 7fd19d600640 1 -- 172.21.15.6:0/3793386010 wait complete. 2024-02-26T06:27:36.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T06:25:17.657061+0000', 'modified': '2024-02-26T06:25:24.777181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24587, 'mds_1': 24569, 'mds_2': 24544, 'mds_3': 24530, 'mds_4': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/66780761', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 66780761}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 66780761}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6833/707593181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6832', 'nonce': 707593181}, {'type': 'v1', 'addr': '172.21.15.90:6833', 'nonce': 707593181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.6:6837/234881158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 234881158}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 234881158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/679395830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 679395830}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 679395830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/2489701568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2489701568}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2489701568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.167:6837/30308127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 30308127}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 30308127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.90:6837/3613516978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 3613516978}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 3613516978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.167:6839/3988004767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3988004767}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3988004767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.6:6841/3799445989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6840', 'nonce': 3799445989}, {'type': 'v1', 'addr': '172.21.15.6:6841', 'nonce': 3799445989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.90:6839/137401855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 137401855}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 137401855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T06:27:36.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-02-26T06:27:36.451 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-26T06:27:36.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:36 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3269211531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:36.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:36 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2227567859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:36.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:36 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3269211531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:36.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:36 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2227567859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 -- 172.21.15.6:0/2794909162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 msgr2=0x7f7830106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 --2- 172.21.15.6:0/2794909162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 0x7f7830106770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f781c009860 tx=0x7f781c02bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 -- 172.21.15.6:0/2794909162 shutdown_connections 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 --2- 172.21.15.6:0/2794909162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 0x7f7830109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 --2- 172.21.15.6:0/2794909162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 0x7f7830106770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 --2- 172.21.15.6:0/2794909162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f7830103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 -- 172.21.15.6:0/2794909162 >> 172.21.15.6:0/2794909162 conn(0x7f78300fb460 msgr2=0x7f78300fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 -- 172.21.15.6:0/2794909162 shutdown_connections 2024-02-26T06:27:36.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.671+0000 7f782f200640 1 -- 172.21.15.6:0/2794909162 wait complete. 2024-02-26T06:27:36.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 Processor -- start 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 -- start start 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f78301013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 0x7f78300ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 0x7f78300fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78301006a0 con 0x7f7830101a50 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7830100810 con 0x7f7830106cb0 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78301a10c0 con 0x7f7830104380 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f78301013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.673+0000 7f782e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 0x7f78300fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f78301013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34902/0 (socket says 172.21.15.6:34902) 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782de00640 1 -- 172.21.15.6:0/1143374254 learned_addr learned my addr 172.21.15.6:0/1143374254 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782d400640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 0x7f78300ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782e800640 1 -- 172.21.15.6:0/1143374254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 msgr2=0x7f78300ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782e800640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 0x7f78300ffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782e800640 1 -- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 msgr2=0x7f78301013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782e800640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f78301013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782e800640 1 -- 172.21.15.6:0/1143374254 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820009560 con 0x7f7830106cb0 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.674+0000 7f782de00640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f78301013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.675+0000 7f782e800640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 0x7f78300fff40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f782800ea80 tx=0x7f782800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.675+0000 7f7826a00640 1 -- 172.21.15.6:0/1143374254 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800cea0 con 0x7f7830106cb0 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.675+0000 7f7826a00640 1 -- 172.21.15.6:0/1143374254 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7828005020 con 0x7f7830106cb0 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.675+0000 7f7826a00640 1 -- 172.21.15.6:0/1143374254 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828010690 con 0x7f7830106cb0 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.675+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c009490 con 0x7f7830106cb0 2024-02-26T06:27:36.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.675+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78301a1670 con 0x7f7830106cb0 2024-02-26T06:27:36.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.678+0000 7f7826a00640 1 -- 172.21.15.6:0/1143374254 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f78280107f0 con 0x7f7830106cb0 2024-02-26T06:27:36.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.678+0000 7f78381fa740 1 Processor -- start 2024-02-26T06:27:36.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.678+0000 7f78381fa740 1 -- start start 2024-02-26T06:27:36.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f78381fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f78381fa740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 0x555c128c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f78381fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f78381fa740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c12830b00 con 0x555c128c2120 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f78381fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c1282e030 con 0x555c128c1730 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f78381fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c12829810 con 0x555c128c7290 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 0x555c128c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 0x555c128c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34910/0 (socket says 172.21.15.6:34910) 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f782de00640 1 -- 172.21.15.6:0/2883064630 learned_addr learned my addr 172.21.15.6:0/2883064630 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f782d400640 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f782d400640 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.679+0000 7f782de00640 1 -- 172.21.15.6:0/2883064630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 msgr2=0x555c128c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782de00640 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782de00640 1 -- 172.21.15.6:0/2883064630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 msgr2=0x555c128c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782de00640 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782de00640 1 -- 172.21.15.6:0/2883064630 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c127f1490 con 0x555c128c2120 2024-02-26T06:27:36.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782d400640 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782de00640 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 0x555c128c6c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7820011810 tx=0x7f7820011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d20b1fa268a989 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f7826a00640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f78140a2140 0x7f78140a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f781b400640 1 -- 172.21.15.6:0/2883064630 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000fba0 con 0x555c128c2120 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f781b400640 1 -- 172.21.15.6:0/2883064630 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f782000fd00 con 0x555c128c2120 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f781b400640 1 -- 172.21.15.6:0/2883064630 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000e680 con 0x555c128c2120 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f7826a00640 1 -- 172.21.15.6:0/1143374254 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7828014070 con 0x7f7830106cb0 2024-02-26T06:27:36.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.680+0000 7f782d400640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f78140a2140 0x7f78140a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 -- 172.21.15.6:0/2883064630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 msgr2=0x555c128c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 0x555c128c6c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7820011810 tx=0x7f7820011c10 comp rx=0 tx=0).stop 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 -- 172.21.15.6:0/2883064630 shutdown_connections 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c2120 0x555c128c6c20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 --2- 172.21.15.6:0/2883064630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 -- 172.21.15.6:0/2883064630 >> 172.21.15.6:0/2883064630 conn(0x555c128bcca0 msgr2=0x555c128bd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 -- 172.21.15.6:0/2883064630 shutdown_connections 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f782d400640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f78140a2140 0x7f78140a4600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f781c009830 tx=0x7f781c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.681+0000 7f78381fa740 1 -- 172.21.15.6:0/2883064630 wait complete. 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.682+0000 7f78381fa740 1 -- start start 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.682+0000 7f78381fa740 1 Processor -- start 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.682+0000 7f78381fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.682+0000 7f78381fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.682+0000 7f78381fa740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 0x555c128d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.682+0000 7f78381fa740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c127cee70 con 0x555c128c7290 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f78381fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c127ceb10 con 0x555c128c1730 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f78381fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c12571f20 con 0x555c128c2120 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f78381fa740 10 client.0 _ll_register_callbacks cb 0x555c127e2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 0x555c128d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 0x555c128d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34920/0 (socket says 172.21.15.6:34920) 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 -- 172.21.15.6:0/1222913091 learned_addr learned my addr 172.21.15.6:0/1222913091 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782de00640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782d400640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 -- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 msgr2=0x555c128cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 -- 172.21.15.6:0/1222913091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 msgr2=0x555c128cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 -- 172.21.15.6:0/1222913091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820011440 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.683+0000 7f782e800640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 0x555c128d2a80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7828097460 tx=0x7f782809b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485f6a4ebefc3706 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f780fe00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f780fe00640 1 -- 172.21.15.6:0/1222913091 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828005190 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f780fe00640 1 -- 172.21.15.6:0/1222913091 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f78280052f0 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c1284dc10 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f780fe00640 1 -- 172.21.15.6:0/1222913091 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78280a3310 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f78381fa740 10 client.15063 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555c12827e90 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f780fe00640 1 -- 172.21.15.6:0/1222913091 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c1284dc10 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.684+0000 7f780fe00640 1 -- 172.21.15.6:0/1222913091 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f782801f070 con 0x555c128c7290 2024-02-26T06:27:36.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f78381fa740 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f78381fa740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:27:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f78381fa740 20 client.15063 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f78381fa740 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c128d7a40 0x555c128dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f78381fa740 4 client.15063 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c125296c0 con 0x555c128d7a40 2024-02-26T06:27:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.685+0000 7f782de00640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c128d7a40 0x555c128dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.686+0000 7f782de00640 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c128d7a40 0x555c128dbec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9038d7b846d42450 server_cookie=6876f34310e0961 in_seq=0 out_seq=0 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.686+0000 7f780fe00640 10 client.15063 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.686+0000 7f780fe00640 1 -- 172.21.15.6:0/1222913091 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c125296c0 con 0x555c128d7a40 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.686+0000 7f780fe00640 10 client.15063 handle_command_reply: tid=0 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.686+0000 7f78381fa740 1 client.15063 shutdown 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 msgr2=0x555c128d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 0x555c128d2a80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7828097460 tx=0x7f782809b0c0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 shutdown_connections 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c128d7a40 0x555c128dbec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c7290 0x555c128d2a80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 --2- 172.21.15.6:0/1222913091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c1730 0x555c128cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 >> 172.21.15.6:0/1222913091 conn(0x555c128bcca0 msgr2=0x555c128bbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f780fe00640 0 client.15063 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 shutdown_connections 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 1 -- 172.21.15.6:0/1222913091 wait complete. 2024-02-26T06:27:36.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.687+0000 7f78381fa740 20 client.15063 trim_cache size 0 max 16384 2024-02-26T06:27:36.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.704+0000 7f78381fa740 1 Processor -- start 2024-02-26T06:27:36.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.704+0000 7f78381fa740 1 -- start start 2024-02-26T06:27:36.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.704+0000 7f78381fa740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.704+0000 7f78381fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f78381fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c7290 0x555c128cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f78381fa740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c125296c0 con 0x555c128c1730 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f78381fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c1284dc10 con 0x555c128c7290 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f78381fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c12822510 con 0x555c128c2120 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34922/0 (socket says 172.21.15.6:34922) 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 -- 172.21.15.6:0/3977873589 learned_addr learned my addr 172.21.15.6:0/3977873589 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782e800640 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c7290 0x555c128cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782d400640 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 -- 172.21.15.6:0/3977873589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 msgr2=0x555c128d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 -- 172.21.15.6:0/3977873589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c7290 msgr2=0x555c128cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c7290 0x555c128cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.705+0000 7f782de00640 1 -- 172.21.15.6:0/3977873589 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c127c4880 con 0x555c128c1730 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.706+0000 7f782de00640 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128ca7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f782001c040 tx=0x7f7820000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21acb36f1bf4ae7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.706+0000 7f781aa00640 1 -- 172.21.15.6:0/3977873589 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000fba0 con 0x555c128c1730 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.706+0000 7f781aa00640 1 -- 172.21.15.6:0/3977873589 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f782002d070 con 0x555c128c1730 2024-02-26T06:27:36.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.706+0000 7f781aa00640 1 -- 172.21.15.6:0/3977873589 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820019430 con 0x555c128c1730 2024-02-26T06:27:36.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 -- 172.21.15.6:0/3977873589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 msgr2=0x555c128ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128ca7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f782001c040 tx=0x7f7820000bc0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 -- 172.21.15.6:0/3977873589 shutdown_connections 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c7290 0x555c128cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c2120 0x555c128d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 --2- 172.21.15.6:0/3977873589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128ca7f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 -- 172.21.15.6:0/3977873589 >> 172.21.15.6:0/3977873589 conn(0x555c128bcca0 msgr2=0x555c128cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 -- 172.21.15.6:0/3977873589 shutdown_connections 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 -- 172.21.15.6:0/3977873589 wait complete. 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 -- start start 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.707+0000 7f78381fa740 1 Processor -- start 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c2120 0x555c128cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c128cfb80 con 0x555c128c1730 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c128cfcf0 con 0x555c128c2120 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c12936390 con 0x555c128c7290 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f78381fa740 10 client.0 _ll_register_callbacks cb 0x7f7828005800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34932/0 (socket says 172.21.15.6:34932) 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 -- 172.21.15.6:0/3914037207 learned_addr learned my addr 172.21.15.6:0/3914037207 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782d400640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c2120 0x555c128cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782e800640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 -- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 msgr2=0x555c128cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 -- 172.21.15.6:0/3914037207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c2120 msgr2=0x555c128cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.708+0000 7f782de00640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c2120 0x555c128cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f782de00640 1 -- 172.21.15.6:0/3914037207 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c127f1490 con 0x555c128c1730 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f782de00640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128d2660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7820011730 tx=0x7f7820005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b768fbaf3d26588e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f780fe00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f780fe00640 1 -- 172.21.15.6:0/3914037207 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200035f0 con 0x555c128c1730 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f780fe00640 1 -- 172.21.15.6:0/3914037207 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7820003750 con 0x555c128c1730 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c128d7750 con 0x555c128c1730 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f780fe00640 1 -- 172.21.15.6:0/3914037207 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7820005030 con 0x555c128c1730 2024-02-26T06:27:36.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f78381fa740 10 client.15069 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.709+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555c127ea4e0 con 0x555c128c1730 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f780fe00640 1 -- 172.21.15.6:0/3914037207 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c128d7750 con 0x555c128c1730 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f780fe00640 1 -- 172.21.15.6:0/3914037207 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7820019aa0 con 0x555c128c1730 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f78381fa740 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f78381fa740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f78381fa740 20 client.15069 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f78381fa740 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c1293b250 0x555c129376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f78381fa740 4 client.15069 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c128d7db0 con 0x555c1293b250 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.710+0000 7f782d400640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c1293b250 0x555c129376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f782d400640 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c1293b250 0x555c129376d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2be77d0e25e90061 server_cookie=7afafb567b2bb0e5 in_seq=0 out_seq=0 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f780fe00640 10 client.15069 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:36.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f780fe00640 1 -- 172.21.15.6:0/3914037207 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555c128d7db0 con 0x555c1293b250 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f780fe00640 10 client.15069 handle_command_reply: tid=0 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 client.15069 shutdown 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 msgr2=0x555c128d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128d2660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7820011730 tx=0x7f7820005d10 comp rx=0 tx=0).stop 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 shutdown_connections 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x555c1293b250 0x555c129376d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c128c7290 0x555c128cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c128c2120 0x555c128cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f780fe00640 0 client.15069 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 --2- 172.21.15.6:0/3914037207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c128c1730 0x555c128d2660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.711+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 >> 172.21.15.6:0/3914037207 conn(0x555c128bcca0 msgr2=0x555c128bbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 shutdown_connections 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f78381fa740 1 -- 172.21.15.6:0/3914037207 wait complete. 2024-02-26T06:27:36.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f78381fa740 20 client.15069 trim_cache size 0 max 16384 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 4, 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24520, 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 132.37423255799999, 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:36.719 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 162.41252183099999 2024-02-26T06:27:36.720 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:36.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f78140a2140 msgr2=0x7f78140a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f78140a2140 0x7f78140a4600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f781c009830 tx=0x7f781c02e040 comp rx=0 tx=0).stop 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 msgr2=0x7f78300fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 0x7f78300fff40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f782800ea80 tx=0x7f782800ef40 comp rx=0 tx=0).stop 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 shutdown_connections 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f78140a2140 0x7f78140a4600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7830106cb0 0x7f78300fff40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7830104380 0x7f78300ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 --2- 172.21.15.6:0/1143374254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830101a50 0x7f78301013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.712+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 >> 172.21.15.6:0/1143374254 conn(0x7f78300fb460 msgr2=0x7f7830102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.713+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 shutdown_connections 2024-02-26T06:27:36.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.713+0000 7f782f200640 1 -- 172.21.15.6:0/1143374254 wait complete. 2024-02-26T06:27:36.728 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-26T06:27:36.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:36 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3269211531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:36.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:36 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2227567859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:36.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 -- 172.21.15.6:0/3554374523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd81021c0 msgr2=0x7f6cd8102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 --2- 172.21.15.6:0/3554374523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd81021c0 0x7f6cd8102640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009900 tx=0x7f6cc002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:36.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 -- 172.21.15.6:0/3554374523 shutdown_connections 2024-02-26T06:27:36.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 --2- 172.21.15.6:0/3554374523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cd8102b80 0x7f6cd8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 --2- 172.21.15.6:0/3554374523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd81021c0 0x7f6cd8102640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 --2- 172.21.15.6:0/3554374523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 -- 172.21.15.6:0/3554374523 >> 172.21.15.6:0/3554374523 conn(0x7f6cd80fc540 msgr2=0x7f6cd80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 -- 172.21.15.6:0/3554374523 shutdown_connections 2024-02-26T06:27:36.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.934+0000 7f6cdd600640 1 -- 172.21.15.6:0/3554374523 wait complete. 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 Processor -- start 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 -- start start 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 0x7f6cd8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cd8102b80 0x7f6cd81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81a0d90 con 0x7f6cd8100fd0 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81a0f00 con 0x7f6cd8102b80 2024-02-26T06:27:36.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.935+0000 7f6cdd600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81a10c0 con 0x7f6cd81021c0 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 0x7f6cd8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41132/0 (socket says 172.21.15.6:41132) 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 0x7f6cd8199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:41044/0 (socket says 172.21.15.6:41044) 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6cd7e00640 1 -- 172.21.15.6:0/2832858873 learned_addr learned my addr 172.21.15.6:0/2832858873 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6cdcc00640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cd8102b80 0x7f6cd81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 -- 172.21.15.6:0/2832858873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cd8102b80 msgr2=0x7f6cd81a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cd8102b80 0x7f6cd81a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 -- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 msgr2=0x7f6cd81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 -- 172.21.15.6:0/2832858873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc0009560 con 0x7f6cd81021c0 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.936+0000 7f6ccf400640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 0x7f6cd8199830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0000c00 tx=0x7f6cc0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.937+0000 7f6cd5600640 1 -- 172.21.15.6:0/2832858873 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0004aa0 con 0x7f6cd81021c0 2024-02-26T06:27:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.937+0000 7f6cd5600640 1 -- 172.21.15.6:0/2832858873 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cc00052d0 con 0x7f6cd81021c0 2024-02-26T06:27:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.937+0000 7f6cd5600640 1 -- 172.21.15.6:0/2832858873 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0003e80 con 0x7f6cd81021c0 2024-02-26T06:27:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.937+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd81a1340 con 0x7f6cd81021c0 2024-02-26T06:27:36.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.937+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd81a1820 con 0x7f6cd81021c0 2024-02-26T06:27:36.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 Processor -- start 2024-02-26T06:27:36.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 -- start start 2024-02-26T06:27:36.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6cd5600640 1 -- 172.21.15.6:0/2832858873 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f6cc0005440 con 0x7f6cd81021c0 2024-02-26T06:27:36.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c32730 0x564137c32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c37c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137ba1b00 con 0x564137c38290 2024-02-26T06:27:36.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137b9f030 con 0x564137c33120 2024-02-26T06:27:36.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.940+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137b9a810 con 0x564137c32730 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6cdcc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c32730 0x564137c32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6cdcc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c32730 0x564137c32b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:41050/0 (socket says 172.21.15.6:41050) 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6cdcc00640 1 -- 172.21.15.6:0/3118930739 learned_addr learned my addr 172.21.15.6:0/3118930739 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c37c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 -- 172.21.15.6:0/3118930739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c32730 msgr2=0x564137c32b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c32730 0x564137c32b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 -- 172.21.15.6:0/3118930739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 msgr2=0x564137c37c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c37c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 -- 172.21.15.6:0/3118930739 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564137b62490 con 0x564137c38290 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.941+0000 7f6ccf400640 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c3a720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00edb50 tx=0x7f6cc00f81a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3492f36268ebc6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6ccd200640 1 -- 172.21.15.6:0/3118930739 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00f89a0 con 0x564137c38290 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6cd5600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6cbc0a20c0 0x7f6cbc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6ccd200640 1 -- 172.21.15.6:0/3118930739 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cc00f0500 con 0x564137c38290 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6ccd200640 1 -- 172.21.15.6:0/3118930739 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00b9050 con 0x564137c38290 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6cd5600640 1 -- 172.21.15.6:0/2832858873 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f6cc00f0090 con 0x7f6cd81021c0 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6cdcc00640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6cbc0a20c0 0x7f6cbc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/3118930739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 msgr2=0x564137c3a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.942+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c3a720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00edb50 tx=0x7f6cc00f81a0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/3118930739 shutdown_connections 2024-02-26T06:27:36.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c3a720 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c37c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/3118930739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c32730 0x564137c32b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/3118930739 >> 172.21.15.6:0/3118930739 conn(0x564137c2dca0 msgr2=0x564137c2e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/3118930739 shutdown_connections 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6cdcc00640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6cbc0a20c0 0x7f6cbc0a4580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x564137c33980 tx=0x7f6cd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/3118930739 wait complete. 2024-02-26T06:27:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 -- start start 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.943+0000 7f6ce1ba3740 1 Processor -- start 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c3c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137b3fe70 con 0x564137c32730 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137b3fb10 con 0x564137c33120 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641378e2f20 con 0x564137c38290 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ce1ba3740 10 client.0 _ll_register_callbacks cb 0x564137b53920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c3c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c3c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41146/0 (socket says 172.21.15.6:41146) 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6cd7e00640 1 -- 172.21.15.6:0/933892206 learned_addr learned my addr 172.21.15.6:0/933892206 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6ccf400640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.944+0000 7f6cdcc00640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cd7e00640 1 -- 172.21.15.6:0/933892206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 msgr2=0x564137c43890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c43890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cd7e00640 1 -- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 msgr2=0x564137c3c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cd7e00640 1 -- 172.21.15.6:0/933892206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd81a1820 con 0x564137c32730 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6ccf400640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c3c310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8012ac0 tx=0x7f6cc8012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b206043d09eb346e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cc5600640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cc5600640 1 -- 172.21.15.6:0/933892206 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8017070 con 0x564137c32730 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cc5600640 1 -- 172.21.15.6:0/933892206 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f6cc8011d90 con 0x564137c32730 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564137bbec10 con 0x564137c32730 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.945+0000 7f6cc5600640 1 -- 172.21.15.6:0/933892206 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc8003970 con 0x564137c32730 2024-02-26T06:27:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564137b98e90 con 0x564137c32730 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6cc5600640 1 -- 172.21.15.6:0/933892206 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564137bbec10 con 0x564137c32730 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6cc5600640 1 -- 172.21.15.6:0/933892206 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6cc801c420 con 0x564137c32730 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 20 client.15081 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137c48980 0x564137c47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 4 client.15081 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.946+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56413789a6c0 con 0x564137c48980 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.947+0000 7f6ccf400640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137c48980 0x564137c47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.947+0000 7f6ccf400640 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137c48980 0x564137c47c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dca3c4ae94ffda66 server_cookie=2f724b87b3b0dfa2 in_seq=0 out_seq=0 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.947+0000 7f6cc5600640 10 client.15081 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6cc5600640 1 -- 172.21.15.6:0/933892206 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56413789a6c0 con 0x564137c48980 2024-02-26T06:27:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6cc5600640 10 client.15081 handle_command_reply: tid=0 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 client.15081 shutdown 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 msgr2=0x564137c3c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c3c310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8012ac0 tx=0x7f6cc8012f80 comp rx=0 tx=0).stop 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 shutdown_connections 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137c48980 0x564137c47c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c43890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/933892206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c3c310 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6cc5600640 0 client.15081 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 >> 172.21.15.6:0/933892206 conn(0x564137c2dca0 msgr2=0x564137c2cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 shutdown_connections 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/933892206 wait complete. 2024-02-26T06:27:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.948+0000 7f6ce1ba3740 20 client.15081 trim_cache size 0 max 16384 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 Processor -- start 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 -- start start 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c32730 0x564137c43f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 0x564137c43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c2c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413789a6c0 con 0x564137c38290 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137bbec10 con 0x564137c32730 2024-02-26T06:27:36.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137b93510 con 0x564137c33120 2024-02-26T06:27:36.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ccf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 0x564137c43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6cdcc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c2c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ccf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 0x564137c43820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:41078/0 (socket says 172.21.15.6:41078) 2024-02-26T06:27:36.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c32730 0x564137c43f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.966+0000 7f6ccf400640 1 -- 172.21.15.6:0/831486674 learned_addr learned my addr 172.21.15.6:0/831486674 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6ccf400640 1 -- 172.21.15.6:0/831486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c32730 msgr2=0x564137c43f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6ccf400640 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c32730 0x564137c43f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6ccf400640 1 -- 172.21.15.6:0/831486674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 msgr2=0x564137c2c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6ccf400640 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c2c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6ccf400640 1 -- 172.21.15.6:0/831486674 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564137b35880 con 0x564137c33120 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6ccf400640 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 0x564137c43820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009900 tx=0x7f6cc0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11fa9873ee73c1a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6cc7e00640 1 -- 172.21.15.6:0/831486674 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00f3c60 con 0x564137c33120 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6cc7e00640 1 -- 172.21.15.6:0/831486674 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cc00bcb80 con 0x564137c33120 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.967+0000 7f6cc7e00640 1 -- 172.21.15.6:0/831486674 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00f0350 con 0x564137c33120 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/831486674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 msgr2=0x564137c43820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 0x564137c43820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009900 tx=0x7f6cc0005740 comp rx=0 tx=0).stop 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/831486674 shutdown_connections 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c38290 0x564137c2c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c33120 0x564137c43820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/831486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c32730 0x564137c43f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/831486674 >> 172.21.15.6:0/831486674 conn(0x564137c2dca0 msgr2=0x564137c50e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/831486674 shutdown_connections 2024-02-26T06:27:36.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/831486674 wait complete. 2024-02-26T06:27:36.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 -- start start 2024-02-26T06:27:36.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.968+0000 7f6ce1ba3740 1 Processor -- start 2024-02-26T06:27:36.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c43340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c40320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137c40a20 con 0x564137c32730 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137c43970 con 0x564137c33120 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564137c43b30 con 0x564137c38290 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ce1ba3740 10 client.0 _ll_register_callbacks cb 0x7f6cc8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c43340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c43340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41162/0 (socket says 172.21.15.6:41162) 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6cd7e00640 1 -- 172.21.15.6:0/1497732224 learned_addr learned my addr 172.21.15.6:0/1497732224 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6cdcc00640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c40320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.969+0000 7f6ccf400640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ccf400640 1 -- 172.21.15.6:0/1497732224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 msgr2=0x564137c40320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ccf400640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c40320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ccf400640 1 -- 172.21.15.6:0/1497732224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 msgr2=0x564137c43340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ccf400640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c43340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ccf400640 1 -- 172.21.15.6:0/1497732224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564137b62490 con 0x564137c33120 2024-02-26T06:27:36.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c43340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ccf400640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3fde0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x564137c444c0 tx=0x7f6cc0003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64517d12b2c3147 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6cc5600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6cc5600640 1 -- 172.21.15.6:0/1497732224 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00f0350 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc00f0500 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6cc5600640 1 -- 172.21.15.6:0/1497732224 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f6cc00bd4b0 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6cc5600640 1 -- 172.21.15.6:0/1497732224 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc00f3c60 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ce1ba3740 10 client.25217 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.970+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564137b5b4e0 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6cc5600640 1 -- 172.21.15.6:0/1497732224 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cc00f0500 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6cc5600640 1 -- 172.21.15.6:0/1497732224 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6cc003c070 con 0x564137c33120 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6ce1ba3740 10 client.25217 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6ce1ba3740 10 client.25217 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6ce1ba3740 20 client.25217 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:36.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137ca1730 0x564137c9f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6ce1ba3740 4 client.25217 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5641378eb270 con 0x564137ca1730 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.971+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137ca1730 0x564137c9f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6cd7e00640 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137ca1730 0x564137c9f5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d1225337048499e8 server_cookie=150f0ad448e4362e in_seq=0 out_seq=0 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6cc5600640 10 client.25217 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6cc5600640 1 -- 172.21.15.6:0/1497732224 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5641378eb270 con 0x564137ca1730 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6cc5600640 10 client.25217 handle_command_reply: tid=0 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6ce1ba3740 1 client.25217 shutdown 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 msgr2=0x564137c3fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.972+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3fde0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x564137c444c0 tx=0x7f6cc0003920 comp rx=0 tx=0).stop 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 shutdown_connections 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x564137ca1730 0x564137c9f5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564137c38290 0x564137c40320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564137c33120 0x564137c3fde0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 --2- 172.21.15.6:0/1497732224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564137c32730 0x564137c43340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6cc5600640 0 client.25217 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 >> 172.21.15.6:0/1497732224 conn(0x564137c2dca0 msgr2=0x564137c2c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 shutdown_connections 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 1 -- 172.21.15.6:0/1497732224 wait complete. 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6ce1ba3740 20 client.25217 trim_cache size 0 max 16384 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:36.977 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 3, 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24530, 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 133.675111598, 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 159.953160397 2024-02-26T06:27:36.978 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6cbc0a20c0 msgr2=0x7f6cbc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6cdd600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6cbc0a20c0 0x7f6cbc0a4580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x564137c33980 tx=0x7f6cd0005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 msgr2=0x7f6cd8199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.973+0000 7f6cdd600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 0x7f6cd8199830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0000c00 tx=0x7f6cc0004270 comp rx=0 tx=0).stop 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 shutdown_connections 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f6cbc0a20c0 0x7f6cbc0a4580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cd8102b80 0x7f6cd81a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd81021c0 0x7f6cd8199830 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 --2- 172.21.15.6:0/2832858873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cd8100fd0 0x7f6cd81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 >> 172.21.15.6:0/2832858873 conn(0x7f6cd80fc540 msgr2=0x7f6cd80733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 shutdown_connections 2024-02-26T06:27:36.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:36.974+0000 7f6cdd600640 1 -- 172.21.15.6:0/2832858873 wait complete. 2024-02-26T06:27:36.992 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 -- 172.21.15.6:0/193313061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 msgr2=0x7fe4f0103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 --2- 172.21.15.6:0/193313061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0103e40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009900 tx=0x7fe4e002bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 -- 172.21.15.6:0/193313061 shutdown_connections 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 --2- 172.21.15.6:0/193313061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f0106cb0 0x7fe4f0109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 --2- 172.21.15.6:0/193313061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 0x7fe4f0106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 --2- 172.21.15.6:0/193313061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0103e40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.208+0000 7fe4f5800640 1 -- 172.21.15.6:0/193313061 >> 172.21.15.6:0/193313061 conn(0x7fe4f00fb460 msgr2=0x7fe4f00fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.209+0000 7fe4f5800640 1 -- 172.21.15.6:0/193313061 shutdown_connections 2024-02-26T06:27:37.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.209+0000 7fe4f5800640 1 -- 172.21.15.6:0/193313061 wait complete. 2024-02-26T06:27:37.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.210+0000 7fe4f5800640 1 Processor -- start 2024-02-26T06:27:37.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.210+0000 7fe4f5800640 1 -- start start 2024-02-26T06:27:37.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 0x7fe4f01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f0106cb0 0x7fe4f01a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f5800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0d70 con 0x7fe4f0104380 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f5800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0ee0 con 0x7fe4f0101a50 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f5800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a10a0 con 0x7fe4f0106cb0 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 0x7fe4f01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 0x7fe4f01997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41188/0 (socket says 172.21.15.6:41188) 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4ef400640 1 -- 172.21.15.6:0/2117006123 learned_addr learned my addr 172.21.15.6:0/2117006123 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f0106cb0 0x7fe4f01a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4efe00640 1 -- 172.21.15.6:0/2117006123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f0106cb0 msgr2=0x7fe4f01a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f0106cb0 0x7fe4f01a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4efe00640 1 -- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 msgr2=0x7fe4f01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 0x7fe4f01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.211+0000 7fe4efe00640 1 -- 172.21.15.6:0/2117006123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e0009560 con 0x7fe4f0101a50 2024-02-26T06:27:37.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.212+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0199280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.212+0000 7fe4ecc00640 1 -- 172.21.15.6:0/2117006123 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0004aa0 con 0x7fe4f0101a50 2024-02-26T06:27:37.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.212+0000 7fe4ecc00640 1 -- 172.21.15.6:0/2117006123 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e00052d0 con 0x7fe4f0101a50 2024-02-26T06:27:37.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.212+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01a1320 con 0x7fe4f0101a50 2024-02-26T06:27:37.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.212+0000 7fe4ecc00640 1 -- 172.21.15.6:0/2117006123 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0003e80 con 0x7fe4f0101a50 2024-02-26T06:27:37.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.212+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01a1850 con 0x7fe4f0101a50 2024-02-26T06:27:37.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.215+0000 7fe4f9c15740 1 Processor -- start 2024-02-26T06:27:37.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.215+0000 7fe4ecc00640 1 -- 172.21.15.6:0/2117006123 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe4e0005440 con 0x7fe4f0101a50 2024-02-26T06:27:37.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 -- start start 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d617730 0x55879d617b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d618120 0x55879d61cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d61f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d586b00 con 0x55879d617730 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d584030 con 0x55879d618120 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d57f810 con 0x55879d61d290 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d617730 0x55879d617b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4f4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d617730 0x55879d617b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41200/0 (socket says 172.21.15.6:41200) 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.216+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d61f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4f4e00640 1 -- 172.21.15.6:0/1726396745 learned_addr learned my addr 172.21.15.6:0/1726396745 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4efe00640 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d618120 0x55879d61cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ef400640 1 -- 172.21.15.6:0/1726396745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d618120 msgr2=0x55879d61cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ef400640 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d618120 0x55879d61cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ef400640 1 -- 172.21.15.6:0/1726396745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d617730 msgr2=0x55879d617b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ef400640 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d617730 0x55879d617b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ef400640 1 -- 172.21.15.6:0/1726396745 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55879d547490 con 0x55879d61d290 2024-02-26T06:27:37.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ecc00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe4d40a2140 0x7fe4d40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ecc00640 1 -- 172.21.15.6:0/2117006123 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe4e00f0090 con 0x7fe4f0101a50 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4ef400640 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d61f720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005bc0 tx=0x7fe4dc004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0c612f5b532b85 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.217+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe4d40a2140 0x7fe4d40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4e5600640 1 -- 172.21.15.6:0/1726396745 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc003da0 con 0x55879d61d290 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4e5600640 1 -- 172.21.15.6:0/1726396745 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc00be50 con 0x55879d61d290 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4e5600640 1 -- 172.21.15.6:0/1726396745 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc011440 con 0x55879d61d290 2024-02-26T06:27:37.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 -- 172.21.15.6:0/1726396745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 msgr2=0x55879d61f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe4d40a2140 0x7fe4d40a4600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55879d618980 tx=0x7fe4e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d61f720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005bc0 tx=0x7fe4dc004610 comp rx=0 tx=0).stop 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 -- 172.21.15.6:0/1726396745 shutdown_connections 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d61f720 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d618120 0x55879d61cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/1726396745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d617730 0x55879d617b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.218+0000 7fe4f9c15740 1 -- 172.21.15.6:0/1726396745 >> 172.21.15.6:0/1726396745 conn(0x55879d612ca0 msgr2=0x55879d6130a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.219+0000 7fe4f9c15740 1 -- 172.21.15.6:0/1726396745 shutdown_connections 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.219+0000 7fe4f9c15740 1 -- 172.21.15.6:0/1726396745 wait complete. 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.219+0000 7fe4f9c15740 1 -- start start 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.219+0000 7fe4f9c15740 1 Processor -- start 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d621560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d621aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d628ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d524e70 con 0x55879d61d290 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d524b10 con 0x55879d617730 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d2c7f20 con 0x55879d618120 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f9c15740 10 client.0 _ll_register_callbacks cb 0x55879d538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d628ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d628ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41204/0 (socket says 172.21.15.6:41204) 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2118656470 learned_addr learned my addr 172.21.15.6:0/2118656470 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d621560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4ef400640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d621aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 msgr2=0x55879d621aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d621aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2118656470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 msgr2=0x55879d621560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.220+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d621560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2118656470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc004270 con 0x55879d61d290 2024-02-26T06:27:37.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d621560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d628ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe4e801f680 tx=0x7fe4e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf3508f36923a7f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4da000640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:37.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4da000640 1 -- 172.21.15.6:0/2118656470 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8016070 con 0x55879d61d290 2024-02-26T06:27:37.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4da000640 1 -- 172.21.15.6:0/2118656470 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe4e8004b20 con 0x55879d61d290 2024-02-26T06:27:37.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55879d5a3c10 con 0x55879d61d290 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.221+0000 7fe4da000640 1 -- 172.21.15.6:0/2118656470 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e8004ce0 con 0x55879d61d290 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55879d57de90 con 0x55879d61d290 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4da000640 1 -- 172.21.15.6:0/2118656470 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55879d5a3c10 con 0x55879d61d290 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4da000640 1 -- 172.21.15.6:0/2118656470 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe4e8030070 con 0x55879d61d290 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 20 client.15099 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d62dc00 0x55879d62ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 4 client.15099 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:37.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.222+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55879d27f6c0 con 0x55879d62dc00 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.223+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d62dc00 0x55879d62ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.223+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d62dc00 0x55879d62ce20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=804ae488e89a3818 server_cookie=715ee60b55281e5c in_seq=0 out_seq=0 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.223+0000 7fe4da000640 10 client.15099 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4da000640 1 -- 172.21.15.6:0/2118656470 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55879d27f6c0 con 0x55879d62dc00 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4da000640 10 client.15099 handle_command_reply: tid=0 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 client.15099 shutdown 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 msgr2=0x55879d628ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d628ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe4e801f680 tx=0x7fe4e8004ab0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 shutdown_connections 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d62dc00 0x55879d62ce20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d628ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d621aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2118656470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d621560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4da000640 0 client.15099 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:37.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 >> 172.21.15.6:0/2118656470 conn(0x55879d612ca0 msgr2=0x55879d611de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 shutdown_connections 2024-02-26T06:27:37.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.224+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2118656470 wait complete. 2024-02-26T06:27:37.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.225+0000 7fe4f9c15740 20 client.15099 trim_cache size 0 max 16384 2024-02-26T06:27:37.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 Processor -- start 2024-02-26T06:27:37.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 -- start start 2024-02-26T06:27:37.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d6360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d628aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d621560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d6173e0 con 0x55879d61d290 2024-02-26T06:27:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d62b9d0 con 0x55879d617730 2024-02-26T06:27:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.242+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d5a3c10 con 0x55879d618120 2024-02-26T06:27:37.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d621560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d621560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41220/0 (socket says 172.21.15.6:41220) 2024-02-26T06:27:37.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2889878408 learned_addr learned my addr 172.21.15.6:0/2889878408 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d6360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4ef400640 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d628aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2889878408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 msgr2=0x55879d628aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d628aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2889878408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 msgr2=0x55879d6360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d6360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 -- 172.21.15.6:0/2889878408 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55879d51a880 con 0x55879d61d290 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4ef400640 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d628aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d621560 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009bc0 tx=0x7fe4e8004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dda4552d293f074 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4e4c00640 1 -- 172.21.15.6:0/2889878408 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80057c0 con 0x55879d61d290 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4e4c00640 1 -- 172.21.15.6:0/2889878408 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8005920 con 0x55879d61d290 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.243+0000 7fe4e4c00640 1 -- 172.21.15.6:0/2889878408 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80053b0 con 0x55879d61d290 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2889878408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 msgr2=0x55879d621560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d621560 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009bc0 tx=0x7fe4e8004510 comp rx=0 tx=0).stop 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2889878408 shutdown_connections 2024-02-26T06:27:37.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d61d290 0x55879d621560 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d618120 0x55879d628aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2889878408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d6360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2889878408 >> 172.21.15.6:0/2889878408 conn(0x55879d612ca0 msgr2=0x55879d60dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2889878408 shutdown_connections 2024-02-26T06:27:37.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.244+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2889878408 wait complete. 2024-02-26T06:27:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 -- start start 2024-02-26T06:27:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 Processor -- start 2024-02-26T06:27:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d628900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d618120 0x55879d6253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d6258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d625f40 con 0x55879d618120 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d68c010 con 0x55879d617730 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879d68c150 con 0x55879d61d290 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4f9c15740 10 client.0 _ll_register_callbacks cb 0x7fe4e8009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d618120 0x55879d6253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.245+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d628900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d618120 0x55879d6253a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41234/0 (socket says 172.21.15.6:41234) 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d628900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:45754/0 (socket says 172.21.15.6:45754) 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4ef400640 1 -- 172.21.15.6:0/2678405728 learned_addr learned my addr 172.21.15.6:0/2678405728 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4f4e00640 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d6258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 -- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 msgr2=0x55879d6258e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d6258e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 -- 172.21.15.6:0/2678405728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d618120 msgr2=0x55879d6253a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d618120 0x55879d6253a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 -- 172.21.15.6:0/2678405728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55879d57de90 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4efe00640 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d628900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00edbd0 tx=0x7fe4e0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa11938e479a6556 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4da000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4da000640 1 -- 172.21.15.6:0/2678405728 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00bd4b0 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4da000640 1 -- 172.21.15.6:0/2678405728 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe4e00bcdb0 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.246+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55879d62d910 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4da000640 1 -- 172.21.15.6:0/2678405728 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e00c0f00 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 10 client.25235 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55879d5404e0 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4da000640 1 -- 172.21.15.6:0/2678405728 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55879d62d910 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4da000640 1 -- 172.21.15.6:0/2678405728 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe4e003c070 con 0x55879d617730 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 10 client.25235 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 10 client.25235 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 20 client.25235 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d690ff0 0x55879d68d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 4 client.25235 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.247+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fe4dc00be50 con 0x55879d690ff0 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.248+0000 7fe4ef400640 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d690ff0 0x55879d68d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.248+0000 7fe4ef400640 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d690ff0 0x55879d68d470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46bda3677d73d71 server_cookie=65b9482db77fd63d in_seq=0 out_seq=0 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.248+0000 7fe4da000640 10 client.25235 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.248+0000 7fe4da000640 1 -- 172.21.15.6:0/2678405728 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fe4dc00be50 con 0x55879d690ff0 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.248+0000 7fe4da000640 10 client.25235 handle_command_reply: tid=0 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 client.25235 shutdown 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 msgr2=0x55879d628900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d628900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00edbd0 tx=0x7fe4e0005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 shutdown_connections 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55879d690ff0 0x55879d68d470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55879d61d290 0x55879d6258e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55879d618120 0x55879d6253a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4da000640 0 client.25235 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 --2- 172.21.15.6:0/2678405728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55879d617730 0x55879d628900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 >> 172.21.15.6:0/2678405728 conn(0x55879d612ca0 msgr2=0x55879d6360a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 shutdown_connections 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 1 -- 172.21.15.6:0/2678405728 wait complete. 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.249+0000 7fe4f9c15740 20 client.25235 trim_cache size 0 max 16384 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24544, 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:37.253 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 134.96331473000001, 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 157.402535384 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:37.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe4d40a2140 msgr2=0x7fe4d40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe4d40a2140 0x7fe4d40a4600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55879d618980 tx=0x7fe4e8009500 comp rx=0 tx=0).stop 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 msgr2=0x7fe4f0199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0199280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e0004270 comp rx=0 tx=0).stop 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 shutdown_connections 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4f0106cb0 0x7fe4f01a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4f0104380 0x7fe4f01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe4d40a2140 0x7fe4d40a4600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 --2- 172.21.15.6:0/2117006123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f0101a50 0x7fe4f0199280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 >> 172.21.15.6:0/2117006123 conn(0x7fe4f00fb460 msgr2=0x7fe4f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 shutdown_connections 2024-02-26T06:27:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.250+0000 7fe4f5800640 1 -- 172.21.15.6:0/2117006123 wait complete. 2024-02-26T06:27:37.268 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-26T06:27:37.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 -- 172.21.15.6:0/700656870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74073be0 msgr2=0x7f4e74074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 --2- 172.21.15.6:0/700656870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74073be0 0x7f4e74074060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009900 tx=0x7f4e5c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 -- 172.21.15.6:0/700656870 shutdown_connections 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 --2- 172.21.15.6:0/700656870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74108e80 0x7f4e7410b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 --2- 172.21.15.6:0/700656870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74073be0 0x7f4e74074060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 --2- 172.21.15.6:0/700656870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 0x7f4e74073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 -- 172.21.15.6:0/700656870 >> 172.21.15.6:0/700656870 conn(0x7f4e740fc530 msgr2=0x7f4e740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 -- 172.21.15.6:0/700656870 shutdown_connections 2024-02-26T06:27:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.492+0000 7f4e73200640 1 -- 172.21.15.6:0/700656870 wait complete. 2024-02-26T06:27:37.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.493+0000 7f4e73200640 1 Processor -- start 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.493+0000 7f4e73200640 1 -- start start 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e73200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 0x7f4e74199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e73200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 0x7f4e741997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e73200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74108e80 0x7f4e741a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e73200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741a0d60 con 0x7f4e740751c0 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e73200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741a0ed0 con 0x7f4e74108e80 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e73200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741a1090 con 0x7f4e74073be0 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e69400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 0x7f4e741997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 0x7f4e74199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 0x7f4e74199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:41138/0 (socket says 172.21.15.6:41138) 2024-02-26T06:27:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e71e00640 1 -- 172.21.15.6:0/3206843489 learned_addr learned my addr 172.21.15.6:0/3206843489 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.494+0000 7f4e72800640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74108e80 0x7f4e741a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e69400640 1 -- 172.21.15.6:0/3206843489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 msgr2=0x7f4e74199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e69400640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 0x7f4e74199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e69400640 1 -- 172.21.15.6:0/3206843489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74108e80 msgr2=0x7f4e741a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e69400640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74108e80 0x7f4e741a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e69400640 1 -- 172.21.15.6:0/3206843489 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64009490 con 0x7f4e740751c0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e71e00640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 0x7f4e74199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e69400640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 0x7f4e741997b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c02c180 tx=0x7f4e5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e6b400640 1 -- 172.21.15.6:0/3206843489 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0044c0 con 0x7f4e740751c0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c009560 con 0x7f4e740751c0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e6b400640 1 -- 172.21.15.6:0/3206843489 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c0052d0 con 0x7f4e740751c0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.495+0000 7f4e6b400640 1 -- 172.21.15.6:0/3206843489 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003e90 con 0x7f4e740751c0 2024-02-26T06:27:37.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.496+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e741a16b0 con 0x7f4e740751c0 2024-02-26T06:27:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 Processor -- start 2024-02-26T06:27:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 -- start start 2024-02-26T06:27:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a2175bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a21760c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21763720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a216cab00 con 0x558a2175c120 2024-02-26T06:27:37.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a216c8030 con 0x558a2175b730 2024-02-26T06:27:37.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a216c3810 con 0x558a21761290 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.499+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a21760c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a2175bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e6b400640 1 -- 172.21.15.6:0/3206843489 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4e5c005440 con 0x7f4e740751c0 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a2175bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:45778/0 (socket says 172.21.15.6:45778) 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 -- 172.21.15.6:0/1753263225 learned_addr learned my addr 172.21.15.6:0/1753263225 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 -- 172.21.15.6:0/1753263225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 msgr2=0x558a21763720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21763720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 -- 172.21.15.6:0/1753263225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 msgr2=0x558a21760c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a21760c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 -- 172.21.15.6:0/1753263225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a2168b490 con 0x558a2175b730 2024-02-26T06:27:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e72800640 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a2175bb50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0039f0 tx=0x7f4e6c003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbda9f60ccc3d4fc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.500+0000 7f4e63400640 1 -- 172.21.15.6:0/1753263225 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c01a470 con 0x558a2175b730 2024-02-26T06:27:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e63400640 1 -- 172.21.15.6:0/1753263225 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c011920 con 0x558a2175b730 2024-02-26T06:27:37.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e63400640 1 -- 172.21.15.6:0/1753263225 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c010a10 con 0x558a2175b730 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e6b400640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4e580a2140 0x7f4e580a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1753263225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 msgr2=0x558a2175bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a2175bb50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0039f0 tx=0x7f4e6c003eb0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1753263225 shutdown_connections 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21763720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a21760c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1753263225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a2175bb50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1753263225 >> 172.21.15.6:0/1753263225 conn(0x558a21756ca0 msgr2=0x558a217570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1753263225 shutdown_connections 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e6b400640 1 -- 172.21.15.6:0/3206843489 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4e5c0f04e0 con 0x7f4e740751c0 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.501+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1753263225 wait complete. 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.502+0000 7f4e71e00640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4e580a2140 0x7f4e580a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.502+0000 7f4e7bc2a740 1 -- start start 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.502+0000 7f4e7bc2a740 1 Processor -- start 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.502+0000 7f4e71e00640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4e580a2140 0x7f4e580a4600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009fd0 tx=0x7f4e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 0x558a21765600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175c120 0x558a21765b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a21761290 0x558a2176cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a21668e70 con 0x558a2175b730 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a21668b10 con 0x558a21761290 2024-02-26T06:27:37.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a2140bf20 con 0x558a2175c120 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e7bc2a740 10 client.0 _ll_register_callbacks cb 0x558a2167c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 0x558a21765600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 0x558a21765600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41278/0 (socket says 172.21.15.6:41278) 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e71e00640 1 -- 172.21.15.6:0/1090100055 learned_addr learned my addr 172.21.15.6:0/1090100055 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e72800640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a21761290 0x558a2176cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.503+0000 7f4e69400640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175c120 0x558a21765b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e71e00640 1 -- 172.21.15.6:0/1090100055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175c120 msgr2=0x558a21765b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e71e00640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175c120 0x558a21765b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e71e00640 1 -- 172.21.15.6:0/1090100055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a21761290 msgr2=0x558a2176cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e71e00640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a21761290 0x558a2176cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e71e00640 1 -- 172.21.15.6:0/1090100055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c009c90 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e69400640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175c120 0x558a21765b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e72800640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a21761290 0x558a2176cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e71e00640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 0x558a21765600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400f8d0 tx=0x7f4e64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706975d075c77b52 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e57e00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e57e00640 1 -- 172.21.15.6:0/1090100055 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64017070 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e57e00640 1 -- 172.21.15.6:0/1090100055 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4e64011bd0 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a216e7c10 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.504+0000 7f4e57e00640 1 -- 172.21.15.6:0/1090100055 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6400fe50 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558a216c1e90 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e57e00640 1 -- 172.21.15.6:0/1090100055 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558a216e7c10 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e57e00640 1 -- 172.21.15.6:0/1090100055 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4e640324a0 con 0x558a2175b730 2024-02-26T06:27:37.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 10 client.15117 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 20 client.15117 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a21771c50 0x558a21770ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 4 client.15117 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a213c36c0 con 0x558a21771c50 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.505+0000 7f4e69400640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a21771c50 0x558a21770ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.506+0000 7f4e69400640 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a21771c50 0x558a21770ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d52a5f7e4d20c136 server_cookie=a19436148c86d25b in_seq=0 out_seq=0 2024-02-26T06:27:37.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.506+0000 7f4e57e00640 10 client.15117 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:37.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e57e00640 1 -- 172.21.15.6:0/1090100055 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a213c36c0 con 0x558a21771c50 2024-02-26T06:27:37.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e57e00640 10 client.15117 handle_command_reply: tid=0 2024-02-26T06:27:37.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 client.15117 shutdown 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 msgr2=0x558a21765600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 0x558a21765600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400f8d0 tx=0x7f4e64004730 comp rx=0 tx=0).stop 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 shutdown_connections 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a21771c50 0x558a21770ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a21761290 0x558a2176cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175c120 0x558a21765b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/1090100055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175b730 0x558a21765600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e57e00640 0 client.15117 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 >> 172.21.15.6:0/1090100055 conn(0x558a21756ca0 msgr2=0x558a21755de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 shutdown_connections 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/1090100055 wait complete. 2024-02-26T06:27:37.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.507+0000 7f4e7bc2a740 20 client.15117 trim_cache size 0 max 16384 2024-02-26T06:27:37.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 Processor -- start 2024-02-26T06:27:37.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 -- start start 2024-02-26T06:27:37.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a21757080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a2176cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21764970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a21771100 con 0x558a2175c120 2024-02-26T06:27:37.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a2176d440 con 0x558a2175b730 2024-02-26T06:27:37.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a21773c30 con 0x558a21761290 2024-02-26T06:27:37.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e69400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a2176cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.525+0000 7f4e69400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a2176cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41286/0 (socket says 172.21.15.6:41286) 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e69400640 1 -- 172.21.15.6:0/2689549448 learned_addr learned my addr 172.21.15.6:0/2689549448 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a21757080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e72800640 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21764970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 -- 172.21.15.6:0/2689549448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 msgr2=0x558a21764970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21764970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 -- 172.21.15.6:0/2689549448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 msgr2=0x558a2176cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a2176cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 -- 172.21.15.6:0/2689549448 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a2165e880 con 0x558a2175b730 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e69400640 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a2176cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e71e00640 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a21757080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e640125c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d652f2d468b461 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e62a00640 1 -- 172.21.15.6:0/2689549448 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64032610 con 0x558a2175b730 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e62a00640 1 -- 172.21.15.6:0/2689549448 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e6400f690 con 0x558a2175b730 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.526+0000 7f4e62a00640 1 -- 172.21.15.6:0/2689549448 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004ad0 con 0x558a2175b730 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/2689549448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 msgr2=0x558a21757080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a21757080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e640125c0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/2689549448 shutdown_connections 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a21761290 0x558a21764970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a2175c120 0x558a2176cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/2689549448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175b730 0x558a21757080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/2689549448 >> 172.21.15.6:0/2689549448 conn(0x558a21756ca0 msgr2=0x558a21760980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/2689549448 shutdown_connections 2024-02-26T06:27:37.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.527+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/2689549448 wait complete. 2024-02-26T06:27:37.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 -- start start 2024-02-26T06:27:37.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 Processor -- start 2024-02-26T06:27:37.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175b730 0x558a2176cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175c120 0x558a21769620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a21761290 0x558a21769bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a217c7850 con 0x558a21761290 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a217c7990 con 0x558a2175c120 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a217c7ad0 con 0x558a2175b730 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.528+0000 7f4e7bc2a740 10 client.0 _ll_register_callbacks cb 0x7f4e64012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e72800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a21761290 0x558a21769bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175b730 0x558a2176cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175c120 0x558a21769620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e72800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a21761290 0x558a21769bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41292/0 (socket says 172.21.15.6:41292) 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e71e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175b730 0x558a2176cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:41180/0 (socket says 172.21.15.6:41180) 2024-02-26T06:27:37.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e72800640 1 -- 172.21.15.6:0/4136647315 learned_addr learned my addr 172.21.15.6:0/4136647315 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 -- 172.21.15.6:0/4136647315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175b730 msgr2=0x558a2176cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175b730 0x558a2176cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 -- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a21761290 msgr2=0x558a21769bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a21761290 0x558a21769bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 -- 172.21.15.6:0/4136647315 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e741a16b0 con 0x558a2175c120 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e69400640 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175c120 0x558a21769620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0c5530 tx=0x7f4e5c0b9560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9401936d97240b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.529+0000 7f4e57e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e57e00640 1 -- 172.21.15.6:0/4136647315 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0c1410 con 0x558a2175c120 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e57e00640 1 -- 172.21.15.6:0/4136647315 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4e5c0c5a60 con 0x558a2175c120 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a21772c80 con 0x558a2175c120 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e57e00640 1 -- 172.21.15.6:0/4136647315 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e5c0c1120 con 0x558a2175c120 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 10 client.25253 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:37.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558a216c1e90 con 0x558a2175c120 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e57e00640 1 -- 172.21.15.6:0/4136647315 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558a21772c80 con 0x558a2175c120 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e57e00640 1 -- 172.21.15.6:0/4136647315 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4e5c03c070 con 0x558a2175c120 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 10 client.25253 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 10 client.25253 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 20 client.25253 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a217ccab0 0x558a217c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 4 client.25253 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.530+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f4e6c011920 con 0x558a217ccab0 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.531+0000 7f4e71e00640 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a217ccab0 0x558a217c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.531+0000 7f4e71e00640 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a217ccab0 0x558a217c8f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9cc6b034fdd8be1 server_cookie=2094d56d143d1d3a in_seq=0 out_seq=0 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.531+0000 7f4e57e00640 10 client.25253 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:37.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e57e00640 1 -- 172.21.15.6:0/4136647315 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f4e6c011920 con 0x558a217ccab0 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e57e00640 10 client.25253 handle_command_reply: tid=0 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 client.25253 shutdown 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175c120 msgr2=0x558a21769620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175c120 0x558a21769620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0c5530 tx=0x7f4e5c0b9560 comp rx=0 tx=0).stop 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 shutdown_connections 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x558a217ccab0 0x558a217c8f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558a21761290 0x558a21769bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2175c120 0x558a21769620 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 --2- 172.21.15.6:0/4136647315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a2175b730 0x558a2176cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 >> 172.21.15.6:0/4136647315 conn(0x558a21756ca0 msgr2=0x558a217607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e57e00640 0 client.25253 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 shutdown_connections 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 1 -- 172.21.15.6:0/4136647315 wait complete. 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.532+0000 7f4e7bc2a740 20 client.25253 trim_cache size 0 max 16384 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24569, 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:37.538 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 136.25755352199999, 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 149.244376426 2024-02-26T06:27:37.539 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4e580a2140 msgr2=0x7f4e580a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4e580a2140 0x7f4e580a4600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009fd0 tx=0x7f4e64005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 msgr2=0x7f4e741997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 0x7f4e741997b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c02c180 tx=0x7f4e5c0040c0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 shutdown_connections 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f4e580a2140 0x7f4e580a4600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e74108e80 0x7f4e741a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e740751c0 0x7f4e741997b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 --2- 172.21.15.6:0/3206843489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e74073be0 0x7f4e74199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 >> 172.21.15.6:0/3206843489 conn(0x7f4e740fc530 msgr2=0x7f4e74104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 shutdown_connections 2024-02-26T06:27:37.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.533+0000 7f4e73200640 1 -- 172.21.15.6:0/3206843489 wait complete. 2024-02-26T06:27:37.549 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:37 smithi006 ceph-mon[31790]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:37.550 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:37 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3793386010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:37.553 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T06:27:37.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:37 smithi090 ceph-mon[35681]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:37.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:37 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3793386010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:37 smithi167 ceph-mon[36229]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:27:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:37 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3793386010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.777+0000 7f2eb0a00640 1 -- 172.21.15.6:0/4251194848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac104380 msgr2=0x7f2eac106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.777+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/4251194848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac104380 0x7f2eac106770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ca0 tx=0x7f2e9802c0f0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 -- 172.21.15.6:0/4251194848 shutdown_connections 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/4251194848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eac106cb0 0x7f2eac109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/4251194848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac104380 0x7f2eac106770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/4251194848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac101a50 0x7f2eac103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 -- 172.21.15.6:0/4251194848 >> 172.21.15.6:0/4251194848 conn(0x7f2eac0fb460 msgr2=0x7f2eac0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 -- 172.21.15.6:0/4251194848 shutdown_connections 2024-02-26T06:27:37.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.778+0000 7f2eb0a00640 1 -- 172.21.15.6:0/4251194848 wait complete. 2024-02-26T06:27:37.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 Processor -- start 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 -- start start 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eac101a50 0x7f2eac1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac104380 0x7f2eac0ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 0x7f2eac0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac1006a0 con 0x7f2eac106cb0 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac100810 con 0x7f2eac101a50 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eb0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac1a10d0 con 0x7f2eac104380 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.779+0000 7f2eab200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 0x7f2eac0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2eaa800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eac101a50 0x7f2eac1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac104380 0x7f2eac0ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2eab200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 0x7f2eac0fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41304/0 (socket says 172.21.15.6:41304) 2024-02-26T06:27:37.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2eab200640 1 -- 172.21.15.6:0/3227293137 learned_addr learned my addr 172.21.15.6:0/3227293137 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 -- 172.21.15.6:0/3227293137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eac101a50 msgr2=0x7f2eac1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eac101a50 0x7f2eac1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 -- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 msgr2=0x7f2eac0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 0x7f2eac0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 -- 172.21.15.6:0/3227293137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98009900 con 0x7f2eac104380 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2eab200640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 0x7f2eac0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.780+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac104380 0x7f2eac0ffa00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e9802c630 tx=0x7f2e98009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.781+0000 7f2ea3400640 1 -- 172.21.15.6:0/3227293137 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980329b0 con 0x7f2eac104380 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.781+0000 7f2ea3400640 1 -- 172.21.15.6:0/3227293137 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e98003e50 con 0x7f2eac104380 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.781+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac1a1320 con 0x7f2eac104380 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.781+0000 7f2ea3400640 1 -- 172.21.15.6:0/3227293137 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98040450 con 0x7f2eac104380 2024-02-26T06:27:37.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.781+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac1a1820 con 0x7f2eac104380 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 Processor -- start 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 -- start start 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf155b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf15b290 0x55cfaf15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf0c4b00 con 0x55cfaf155730 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf0c2030 con 0x55cfaf156120 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.784+0000 7f2eb423b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf0bd810 con 0x55cfaf15b290 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eaa800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41306/0 (socket says 172.21.15.6:41306) 2024-02-26T06:27:37.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 -- 172.21.15.6:0/165963150 learned_addr learned my addr 172.21.15.6:0/165963150 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 -- 172.21.15.6:0/165963150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf15b290 msgr2=0x55cfaf15d720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf15b290 0x55cfaf15d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 -- 172.21.15.6:0/165963150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 msgr2=0x55cfaf15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.785+0000 7f2eab200640 1 -- 172.21.15.6:0/165963150 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfaf085490 con 0x55cfaf155730 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2eab200640 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf155b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40115a0 tx=0x7f2ea4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81a9d1a96196751 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2e97400640 1 -- 172.21.15.6:0/165963150 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400f660 con 0x55cfaf155730 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2e97400640 1 -- 172.21.15.6:0/165963150 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ea400fca0 con 0x55cfaf155730 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2e97400640 1 -- 172.21.15.6:0/165963150 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4019d80 con 0x55cfaf155730 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2ea3400640 1 -- 172.21.15.6:0/3227293137 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2e9800a040 con 0x7f2eac104380 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2eb423b740 1 -- 172.21.15.6:0/165963150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 msgr2=0x55cfaf155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.786+0000 7f2eb423b740 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf155b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40115a0 tx=0x7f2ea4011a60 comp rx=0 tx=0).stop 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 -- 172.21.15.6:0/165963150 shutdown_connections 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf15b290 0x55cfaf15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 --2- 172.21.15.6:0/165963150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf155b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 -- 172.21.15.6:0/165963150 >> 172.21.15.6:0/165963150 conn(0x55cfaf150ca0 msgr2=0x55cfaf1510a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 -- 172.21.15.6:0/165963150 shutdown_connections 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 -- 172.21.15.6:0/165963150 wait complete. 2024-02-26T06:27:37.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 -- start start 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2eb423b740 1 Processor -- start 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.787+0000 7f2ea3400640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2e900a2050 0x7f2e900a4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2ea3400640 1 -- 172.21.15.6:0/3227293137 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2e980f0520 con 0x7f2eac104380 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eab200640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2e900a2050 0x7f2e900a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf15b290 0x55cfaf166b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf062e70 con 0x55cfaf155730 2024-02-26T06:27:37.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf062b10 con 0x55cfaf15b290 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaee05f20 con 0x55cfaf156120 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eb423b740 10 client.0 _ll_register_callbacks cb 0x55cfaf076920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eaa800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eaa800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf15f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41322/0 (socket says 172.21.15.6:41322) 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2eaa800640 1 -- 172.21.15.6:0/1873033978 learned_addr learned my addr 172.21.15.6:0/1873033978 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.788+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eab200640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf15b290 0x55cfaf166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eaa800640 1 -- 172.21.15.6:0/1873033978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 msgr2=0x55cfaf15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eab200640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2e900a2050 0x7f2e900a4510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4010010 tx=0x7f2ea4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eaa800640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf15fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eaa800640 1 -- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf15b290 msgr2=0x55cfaf166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eaa800640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf15b290 0x55cfaf166b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eaa800640 1 -- 172.21.15.6:0/1873033978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea4009e20 con 0x55cfaf155730 2024-02-26T06:27:37.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2eaa800640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf15f5d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c012700 tx=0x7f2e9c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c05445e2b843857 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2e8be00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.789+0000 7f2e8be00640 1 -- 172.21.15.6:0/1873033978 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c017070 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2e8be00640 1 -- 172.21.15.6:0/1873033978 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2e9c004020 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfaf0e1c10 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2e8be00640 1 -- 172.21.15.6:0/1873033978 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9c0041e0 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2eb423b740 10 client.15135 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cfaf0bbe90 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2e8be00640 1 -- 172.21.15.6:0/1873033978 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cfaf0e1c10 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2e8be00640 1 -- 172.21.15.6:0/1873033978 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e9c003880 con 0x55cfaf155730 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2eb423b740 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.790+0000 7f2eb423b740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.791+0000 7f2eb423b740 20 client.15135 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.791+0000 7f2eb423b740 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf16b5b0 0x55cfaf170050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.791+0000 7f2eb423b740 4 client.15135 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.791+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfaedbd6c0 con 0x55cfaf16b5b0 2024-02-26T06:27:37.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.791+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf16b5b0 0x55cfaf170050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf16b5b0 0x55cfaf170050 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf76c940c1559370 server_cookie=11d906b7372be9db in_seq=0 out_seq=0 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2e8be00640 10 client.15135 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2e8be00640 1 -- 172.21.15.6:0/1873033978 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfaedbd6c0 con 0x55cfaf16b5b0 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2e8be00640 10 client.15135 handle_command_reply: tid=0 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2eb423b740 1 client.15135 shutdown 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 msgr2=0x55cfaf15f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.792+0000 7f2eb423b740 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf15f5d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c012700 tx=0x7f2e9c012bc0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 shutdown_connections 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf16b5b0 0x55cfaf170050 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf15b290 0x55cfaf166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 --2- 172.21.15.6:0/1873033978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf15f5d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2e8be00640 0 client.15135 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 >> 172.21.15.6:0/1873033978 conn(0x55cfaf150ca0 msgr2=0x55cfaf14fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 shutdown_connections 2024-02-26T06:27:37.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 1 -- 172.21.15.6:0/1873033978 wait complete. 2024-02-26T06:27:37.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.793+0000 7f2eb423b740 20 client.15135 trim_cache size 0 max 16384 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.811+0000 7f2eb423b740 1 Processor -- start 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 -- start start 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf150ca0 0x55cfaf174070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf166a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf1745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaedbd6c0 con 0x55cfaf155730 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf0e1c10 con 0x55cfaf156120 2024-02-26T06:27:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eb423b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf0b6510 con 0x55cfaf150ca0 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf166a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf166a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41324/0 (socket says 172.21.15.6:41324) 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2ea9e00640 1 -- 172.21.15.6:0/100238482 learned_addr learned my addr 172.21.15.6:0/100238482 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.812+0000 7f2eab200640 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf1745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2eaa800640 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf150ca0 0x55cfaf174070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2ea9e00640 1 -- 172.21.15.6:0/100238482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf150ca0 msgr2=0x55cfaf174070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf150ca0 0x55cfaf174070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2ea9e00640 1 -- 172.21.15.6:0/100238482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 msgr2=0x55cfaf1745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf1745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2ea9e00640 1 -- 172.21.15.6:0/100238482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfaf16b2a0 con 0x55cfaf155730 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2eaa800640 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf150ca0 0x55cfaf174070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:37.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf166a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2e980b9c50 tx=0x7f2e980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb408577a4f6d08c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2e96a00640 1 -- 172.21.15.6:0/100238482 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980c1420 con 0x55cfaf155730 2024-02-26T06:27:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2e96a00640 1 -- 172.21.15.6:0/100238482 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e980c59e0 con 0x55cfaf155730 2024-02-26T06:27:37.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.813+0000 7f2e96a00640 1 -- 172.21.15.6:0/100238482 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980bd530 con 0x55cfaf155730 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 -- 172.21.15.6:0/100238482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 msgr2=0x55cfaf166a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf166a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2e980b9c50 tx=0x7f2e980095c0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 -- 172.21.15.6:0/100238482 shutdown_connections 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf156120 0x55cfaf1745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf166a90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 --2- 172.21.15.6:0/100238482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf150ca0 0x55cfaf174070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 -- 172.21.15.6:0/100238482 >> 172.21.15.6:0/100238482 conn(0x55cfaf1513f0 msgr2=0x55cfaf14fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 -- 172.21.15.6:0/100238482 shutdown_connections 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.814+0000 7f2eb423b740 1 -- 172.21.15.6:0/100238482 wait complete. 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 -- start start 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 Processor -- start 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf150ca0 0x55cfaf1669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf163490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf1639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf1640d0 con 0x55cfaf155730 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf1c1d10 con 0x55cfaf150ca0 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaf1c1ed0 con 0x55cfaf156120 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.815+0000 7f2eb423b740 10 client.0 _ll_register_callbacks cb 0x7f2e9c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf163490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41326/0 (socket says 172.21.15.6:41326) 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf1639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2ea9e00640 1 -- 172.21.15.6:0/154608043 learned_addr learned my addr 172.21.15.6:0/154608043 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 -- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf150ca0 msgr2=0x55cfaf1669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf150ca0 0x55cfaf1669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 -- 172.21.15.6:0/154608043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 msgr2=0x55cfaf163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 -- 172.21.15.6:0/154608043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac1a1820 con 0x55cfaf156120 2024-02-26T06:27:37.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2ea9e00640 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2eab200640 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf1639d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ea401c040 tx=0x7f2ea4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e82db9e08376155 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.816+0000 7f2e8be00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2e8be00640 1 -- 172.21.15.6:0/154608043 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4005690 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2e8be00640 1 -- 172.21.15.6:0/154608043 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2ea4004680 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfaf16c5a0 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2e8be00640 1 -- 172.21.15.6:0/154608043 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ea4004840 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2eb423b740 10 client.25252 fetch_fsmap learned FSMap version 26 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cfaf058880 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.817+0000 7f2e8be00640 1 -- 172.21.15.6:0/154608043 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cfaf16c5a0 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2e8be00640 1 -- 172.21.15.6:0/154608043 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2ea4039070 con 0x55cfaf156120 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eb423b740 10 client.25252 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eb423b740 10 client.25252 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eb423b740 20 client.25252 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:37.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eb423b740 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf15b290 0x55cfaf1c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eb423b740 4 client.25252 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T06:27:37.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f2ea400fca0 con 0x55cfaf15b290 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.818+0000 7f2eaa800640 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf15b290 0x55cfaf1c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eaa800640 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf15b290 0x55cfaf1c3650 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e7d888f8d41151 server_cookie=5c22dcf3f9261fa3 in_seq=0 out_seq=0 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2e8be00640 10 client.25252 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2e8be00640 1 -- 172.21.15.6:0/154608043 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f2ea400fca0 con 0x55cfaf15b290 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2e8be00640 10 client.25252 handle_command_reply: tid=0 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 client.25252 shutdown 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 msgr2=0x55cfaf1639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf1639d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ea401c040 tx=0x7f2ea4005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:37.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 shutdown_connections 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55cfaf15b290 0x55cfaf1c3650 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfaf156120 0x55cfaf1639d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cfaf155730 0x55cfaf163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 --2- 172.21.15.6:0/154608043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfaf150ca0 0x55cfaf1669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2e8be00640 0 client.25252 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 >> 172.21.15.6:0/154608043 conn(0x55cfaf1513f0 msgr2=0x55cfaf174070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 shutdown_connections 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.819+0000 7f2eb423b740 1 -- 172.21.15.6:0/154608043 wait complete. 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb423b740 20 client.25252 trim_cache size 0 max 16384 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "5e92f702-d46e-11ee-95c3-87774f69a715", 2024-02-26T06:27:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24587, 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 139.58790414800001, 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 26, 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 85, 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 146.79928161500001 2024-02-26T06:27:37.824 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2e900a2050 msgr2=0x7f2e900a4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2e900a2050 0x7f2e900a4510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4010010 tx=0x7f2ea4000bc0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac104380 msgr2=0x7f2eac0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac104380 0x7f2eac0ffa00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e9802c630 tx=0x7f2e98009360 comp rx=0 tx=0).stop 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 shutdown_connections 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2eac106cb0 0x7f2eac0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eac104380 0x7f2eac0ffa00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f2e900a2050 0x7f2e900a4510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 --2- 172.21.15.6:0/3227293137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eac101a50 0x7f2eac1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 >> 172.21.15.6:0/3227293137 conn(0x7f2eac0fb460 msgr2=0x7f2eac102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:37.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 shutdown_connections 2024-02-26T06:27:37.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:37.820+0000 7f2eb0a00640 1 -- 172.21.15.6:0/3227293137 wait complete. 2024-02-26T06:27:37.838 INFO:teuthology.run_tasks:Running task kclient... 2024-02-26T06:27:37.862 INFO:tasks.kclient:Mounting kernel clients... 2024-02-26T06:27:37.862 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-26T06:27:37.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:37.863 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-02-26T06:27:37.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:37.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-02-26T06:27:38.020 INFO:teuthology.orchestra.run.smithi006.stderr:Cannot find device "ceph-brx" 2024-02-26T06:27:38.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:27:38.026 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-02-26T06:27:38.026 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T06:27:38.026 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T06:27:38.026 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-26T06:27:38.511 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:27:39.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:39 smithi090 ceph-mon[35681]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:39.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:39 smithi006 ceph-mon[31790]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:39.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:39 smithi167 ceph-mon[36229]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 -- 172.21.15.6:0/1356787361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 msgr2=0x7fe2bc06a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1356787361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 0x7fe2bc06a2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009990 tx=0x7fe2a8009ca0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 -- 172.21.15.6:0/1356787361 shutdown_connections 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1356787361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1356787361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2bc106b60 0x7fe2bc069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1356787361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 0x7fe2bc06a2c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 -- 172.21.15.6:0/1356787361 >> 172.21.15.6:0/1356787361 conn(0x7fe2bc077aa0 msgr2=0x7fe2bc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 -- 172.21.15.6:0/1356787361 shutdown_connections 2024-02-26T06:27:40.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.686+0000 7fe2c4200700 1 -- 172.21.15.6:0/1356787361 wait complete. 2024-02-26T06:27:40.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.687+0000 7fe2c4200700 1 Processor -- start 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.687+0000 7fe2c4200700 1 -- start start 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c4200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 0x7fe2bc1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c4200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c4200700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2bc106b60 0x7fe2bc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c4200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19ee40 con 0x7fe2bc104670 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c4200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19efb0 con 0x7fe2bc069e60 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c4200700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19f170 con 0x7fe2bc106b60 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c0e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c0e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc1a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41338/0 (socket says 172.21.15.6:41338) 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 0x7fe2bc1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c0e00700 1 -- 172.21.15.6:0/1365511465 learned_addr learned my addr 172.21.15.6:0/1365511465 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:40.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.688+0000 7fe2c2200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2bc106b60 0x7fe2bc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:40.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c0e00700 1 -- 172.21.15.6:0/1365511465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2bc106b60 msgr2=0x7fe2bc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:40.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c0e00700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2bc106b60 0x7fe2bc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c0e00700 1 -- 172.21.15.6:0/1365511465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 msgr2=0x7fe2bc1a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:40.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c0e00700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 0x7fe2bc1a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c0e00700 1 -- 172.21.15.6:0/1365511465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4009c40 con 0x7fe2bc104670 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c0e00700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc1a4b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a801d070 con 0x7fe2bc104670 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2a800b610 con 0x7fe2bc104670 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80173f0 con 0x7fe2bc104670 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.689+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a80095f0 con 0x7fe2bc104670 2024-02-26T06:27:40.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.690+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1ab370 con 0x7fe2bc104670 2024-02-26T06:27:40.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.691+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc101150 con 0x7fe2bc104670 2024-02-26T06:27:40.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.693+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe2a8022600 con 0x7fe2bc104670 2024-02-26T06:27:40.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.695+0000 7fe2ba000700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2a40a2150 0x7fe2a40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:40.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.695+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe2a80cf740 con 0x7fe2bc104670 2024-02-26T06:27:40.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.695+0000 7fe2c1800700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2a40a2150 0x7fe2a40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:40.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.696+0000 7fe2c1800700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2a40a2150 0x7fe2a40a4610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009be0 tx=0x7fe2b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:40.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.698+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2a8098730 con 0x7fe2bc104670 2024-02-26T06:27:40.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.968+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2bc19fbc0 con 0x7fe2bc104670 2024-02-26T06:27:40.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.969+0000 7fe2ba000700 1 -- 172.21.15.6:0/1365511465 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2a809bdf0 con 0x7fe2bc104670 2024-02-26T06:27:40.970 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T06:27:40.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2a40a2150 msgr2=0x7fe2a40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2a40a2150 0x7fe2a40a4610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009be0 tx=0x7fe2b0009260 comp rx=0 tx=0).stop 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 msgr2=0x7fe2bc1a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc1a4b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8005740 comp rx=0 tx=0).stop 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 shutdown_connections 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fe2a40a2150 0x7fe2a40a4610 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2bc106b60 0x7fe2bc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2bc069e60 0x7fe2bc1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 --2- 172.21.15.6:0/1365511465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc104670 0x7fe2bc1a4b00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:40.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.972+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 >> 172.21.15.6:0/1365511465 conn(0x7fe2bc077aa0 msgr2=0x7fe2bc103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:40.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.973+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 shutdown_connections 2024-02-26T06:27:40.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:40.973+0000 7fe2c4200700 1 -- 172.21.15.6:0/1365511465 wait complete. 2024-02-26T06:27:41.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:41 smithi090 ceph-mon[35681]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:41.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:41 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1365511465' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T06:27:41.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:41 smithi006 ceph-mon[31790]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:41.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:41 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1365511465' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T06:27:41.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:41 smithi167 ceph-mon[36229]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-02-26T06:27:41.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:41 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1365511465' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-26T06:27:41.846 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T06:27:41.847 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T06:27:41.847 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T06:27:41.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:41.847 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-02-26T06:27:41.901 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 41658sec preferred_lft 41658sec 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-02-26T06:27:41.902 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-02-26T06:27:41.902 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T06:27:41.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T06:27:41.903 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-02-26T06:27:42.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:42.357 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T06:27:42.431 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-02-26T06:27:42.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:42.435 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-02-26T06:27:42.486 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-02-26T06:27:42.486 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T06:27:42.487 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-26T06:27:42.487 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-26T06:27:42.487 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T06:27:42.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:42.488 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-02-26T06:27:42.488 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-02-26T06:27:42.488 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-26T06:27:42.488 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-26T06:27:42.489 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-26T06:27:42.489 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-02-26T06:27:43.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:43.204 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-02-26T06:27:43.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:43.264 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-02-26T06:27:43.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:43.322 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-02-26T06:27:43.322 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-02-26T06:27:43.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T06:27:43.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-26T06:27:43.323 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-02-26T06:27:43.454 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:43 smithi006 ceph-mon[31790]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:43.468 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-26T06:27:43.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-26T06:27:43.468 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-02-26T06:27:43.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:43 smithi090 ceph-mon[35681]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:43.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:43 smithi167 ceph-mon[36229]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-02-26T06:27:43.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:43.757 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2024-02-26T06:27:43.757 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-02-26T06:27:43.757 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2024-02-26T06:27:43.757 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T06:27:43.757 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-02-26T06:27:43.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T06:27:43.923 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T06:27:43.972 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T06:27:43.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T06:27:43.973 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T06:27:44.029 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-26T06:27:44.029 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-26T06:27:44.029 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018 2024-02-26T06:27:44.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:44.029 DEBUG:teuthology.orchestra.run.smithi006:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-02-26T06:27:44.184 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-02-26T06:27:44.184 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-02-26T06:27:44.184 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-02-26T06:27:44.881 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-02-26T06:27:44.882 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-02-26T06:27:44.882 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-02-26T06:27:44.882 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.90:3300,172.21.15.167:3300" 2024-02-26T06:27:44.882 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.6:3300,172.21.15.90:3300,172.21.15.167:3300:/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018 2024-02-26T06:27:44.882 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=5e92f702-d46e-11ee-95c3-87774f69a715" will pass to kernel 2024-02-26T06:27:44.890 INFO:teuthology.orchestra.run.smithi006.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T06:27:44.890 INFO:teuthology.orchestra.run.smithi006.stdout: You just mounted a file system that supports labels which does not 2024-02-26T06:27:44.890 INFO:teuthology.orchestra.run.smithi006.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T06:27:44.890 INFO:teuthology.orchestra.run.smithi006.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T06:27:44.890 INFO:teuthology.orchestra.run.smithi006.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T06:27:44.897 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T06:27:44.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:44.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T06:27:44.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:44.997 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T06:27:45.057 INFO:teuthology.orchestra.run.smithi006.stdout:client25277 2024-02-26T06:27:45.058 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:27:45.058 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:27:45.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:45.114 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T06:27:45.168 INFO:teuthology.orchestra.run.smithi006.stdout:client25277 2024-02-26T06:27:45.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:45.169 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T06:27:45.227 INFO:teuthology.orchestra.run.smithi006.stdout:client25277 2024-02-26T06:27:45.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:45.228 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5e92f702-d46e-11ee-95c3-87774f69a715.client25277/status') 2024-02-26T06:27:45.325 INFO:teuthology.orchestra.run.smithi006.stdout:instance: client.25277 (3)172.21.15.6:0/1559277123 2024-02-26T06:27:45.326 INFO:teuthology.orchestra.run.smithi006.stdout:blocklisted: false 2024-02-26T06:27:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records in 2024-02-26T06:27:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records out 2024-02-26T06:27:45.326 INFO:teuthology.orchestra.run.smithi006.stderr:70 bytes copied, 8.2925e-05 s, 844 kB/s 2024-02-26T06:27:45.330 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T06:27:45.341 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:27:45.341 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:27:45.341 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-02-26T06:27:45.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 -- 172.21.15.6:0/4241141586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c1043a0 msgr2=0x7f517c106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 --2- 172.21.15.6:0/4241141586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c1043a0 0x7f517c106790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5168009900 tx=0x7f516802bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 -- 172.21.15.6:0/4241141586 shutdown_connections 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 --2- 172.21.15.6:0/4241141586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c106cd0 0x7f517c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 --2- 172.21.15.6:0/4241141586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c1043a0 0x7f517c106790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 --2- 172.21.15.6:0/4241141586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c101a70 0x7f517c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 -- 172.21.15.6:0/4241141586 >> 172.21.15.6:0/4241141586 conn(0x7f517c0fb440 msgr2=0x7f517c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 -- 172.21.15.6:0/4241141586 shutdown_connections 2024-02-26T06:27:45.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.566+0000 7f5181a00640 1 -- 172.21.15.6:0/4241141586 wait complete. 2024-02-26T06:27:45.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.567+0000 7f5181a00640 1 Processor -- start 2024-02-26T06:27:45.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.567+0000 7f5181a00640 1 -- start start 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 0x7f517c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c1043a0 0x7f517c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c106cd0 0x7f517c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c19eb60 con 0x7f517c101a70 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c19ecd0 con 0x7f517c1043a0 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c19ee90 con 0x7f517c106cd0 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 0x7f517c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 0x7f517c197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41362/0 (socket says 172.21.15.6:41362) 2024-02-26T06:27:45.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 -- 172.21.15.6:0/2689993171 learned_addr learned my addr 172.21.15.6:0/2689993171 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f5181000640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c106cd0 0x7f517c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517b400640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c1043a0 0x7f517c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 -- 172.21.15.6:0/2689993171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c106cd0 msgr2=0x7f517c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c106cd0 0x7f517c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 -- 172.21.15.6:0/2689993171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c1043a0 msgr2=0x7f517c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c1043a0 0x7f517c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517be00640 1 -- 172.21.15.6:0/2689993171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168009560 con 0x7f517c101a70 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.568+0000 7f517b400640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c1043a0 0x7f517c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.569+0000 7f517be00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 0x7f517c197070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e9f0 tx=0x7f516c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.569+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c009700 con 0x7f517c101a70 2024-02-26T06:27:45.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.569+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c19f170 con 0x7f517c101a70 2024-02-26T06:27:45.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.569+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f516c00cd80 con 0x7f517c101a70 2024-02-26T06:27:45.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.569+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c018920 con 0x7f517c101a70 2024-02-26T06:27:45.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.569+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c19f6a0 con 0x7f517c101a70 2024-02-26T06:27:45.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.570+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c0fcf30 con 0x7f517c101a70 2024-02-26T06:27:45.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.572+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f516c018a80 con 0x7f517c101a70 2024-02-26T06:27:45.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.573+0000 7f5178c00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f51600a20a0 0x7f51600a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:45.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.574+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f516c014070 con 0x7f517c101a70 2024-02-26T06:27:45.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.574+0000 7f517b400640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f51600a20a0 0x7f51600a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:45.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.575+0000 7f517b400640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f51600a20a0 0x7f51600a4560 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5168006010 tx=0x7f5168005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:45.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.578+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f516c052d20 con 0x7f517c101a70 2024-02-26T06:27:45.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:45 smithi090 ceph-mon[35681]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:45.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:45 smithi006 ceph-mon[31790]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:45 smithi167 ceph-mon[36229]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:27:45.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:45.823+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f517c02cdd0 con 0x7f517c101a70 2024-02-26T06:27:46.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.348+0000 7f5178c00640 1 -- 172.21.15.6:0/2689993171 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f516c097090 con 0x7f517c101a70 2024-02-26T06:27:46.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f51600a20a0 msgr2=0x7f51600a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f51600a20a0 0x7f51600a4560 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5168006010 tx=0x7f5168005be0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 msgr2=0x7f517c197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 0x7f517c197070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e9f0 tx=0x7f516c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 shutdown_connections 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c106cd0 0x7f517c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f51600a20a0 0x7f51600a4560 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f517c1043a0 0x7f517c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 --2- 172.21.15.6:0/2689993171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c101a70 0x7f517c197070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.352+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 >> 172.21.15.6:0/2689993171 conn(0x7f517c0fb440 msgr2=0x7f517c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.353+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 shutdown_connections 2024-02-26T06:27:46.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.353+0000 7f5181a00640 1 -- 172.21.15.6:0/2689993171 wait complete. 2024-02-26T06:27:46.374 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.586+0000 7f1754400640 1 -- 172.21.15.6:0/3561322152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1089f0 msgr2=0x7f174c108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.586+0000 7f1754400640 1 --2- 172.21.15.6:0/3561322152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1089f0 0x7f174c108e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1738009900 tx=0x7f173802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 -- 172.21.15.6:0/3561322152 shutdown_connections 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 --2- 172.21.15.6:0/3561322152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1093b0 0x7f174c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 --2- 172.21.15.6:0/3561322152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1089f0 0x7f174c108e70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 --2- 172.21.15.6:0/3561322152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 0x7f174c107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 -- 172.21.15.6:0/3561322152 >> 172.21.15.6:0/3561322152 conn(0x7f174c0767d0 msgr2=0x7f174c078bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 -- 172.21.15.6:0/3561322152 shutdown_connections 2024-02-26T06:27:46.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.587+0000 7f1754400640 1 -- 172.21.15.6:0/3561322152 wait complete. 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 Processor -- start 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 -- start start 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 0x7f174c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 0x7f174c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1093b0 0x7f174c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a53c0 con 0x7f174c1089f0 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a5530 con 0x7f174c1093b0 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1754400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a56f0 con 0x7f174c107800 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.588+0000 7f1751000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 0x7f174c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 0x7f174c19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41388/0 (socket says 172.21.15.6:41388) 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751000640 1 -- 172.21.15.6:0/2270806163 learned_addr learned my addr 172.21.15.6:0/2270806163 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 0x7f174c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1752400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1093b0 0x7f174c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 -- 172.21.15.6:0/2270806163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1093b0 msgr2=0x7f174c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1093b0 0x7f174c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 -- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 msgr2=0x7f174c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 0x7f174c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 -- 172.21.15.6:0/2270806163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744009840 con 0x7f174c107800 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751000640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 0x7f174c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.589+0000 7f1751a00640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 0x7f174c19d8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f174000e9c0 tx=0x7f174000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.590+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174000ccc0 con 0x7f174c107800 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.590+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f174000ce20 con 0x7f174c107800 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.590+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738009560 con 0x7f174c107800 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.590+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740010430 con 0x7f174c107800 2024-02-26T06:27:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.590+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1a5d00 con 0x7f174c107800 2024-02-26T06:27:46.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.591+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c10eb40 con 0x7f174c107800 2024-02-26T06:27:46.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.593+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1740003fd0 con 0x7f174c107800 2024-02-26T06:27:46.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.594+0000 7f174a000640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f17340a20b0 0x7f17340a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:46.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.594+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1740014070 con 0x7f174c107800 2024-02-26T06:27:46.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.594+0000 7f1751000640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f17340a20b0 0x7f17340a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:46.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.595+0000 7f1751000640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f17340a20b0 0x7f17340a4570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f173802cdd0 tx=0x7f173802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.598+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17400970f0 con 0x7f174c107800 2024-02-26T06:27:46.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:46 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2689993171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T06:27:46.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:46 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2689993171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T06:27:46.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:46 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2689993171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T06:27:46.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.827+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f174c02ce40 con 0x7f17340a20b0 2024-02-26T06:27:46.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.854+0000 7f174a000640 1 -- 172.21.15.6:0/2270806163 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f174c02ce40 con 0x7f17340a20b0 2024-02-26T06:27:46.855 INFO:teuthology.orchestra.run.smithi006.stdout:{} 2024-02-26T06:27:46.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f17340a20b0 msgr2=0x7f17340a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f17340a20b0 0x7f17340a4570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f173802cdd0 tx=0x7f173802c7e0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 msgr2=0x7f174c19d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 0x7f174c19d8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f174000e9c0 tx=0x7f174000edc0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 shutdown_connections 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f17340a20b0 0x7f17340a4570 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c1093b0 0x7f174c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f174c1089f0 0x7f174c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 --2- 172.21.15.6:0/2270806163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c107800 0x7f174c19d8d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.857+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 >> 172.21.15.6:0/2270806163 conn(0x7f174c0767d0 msgr2=0x7f174c10bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.858+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 shutdown_connections 2024-02-26T06:27:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:46.858+0000 7f1754400640 1 -- 172.21.15.6:0/2270806163 wait complete. 2024-02-26T06:27:46.876 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-26T06:27:46.887 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-26T06:27:46.928 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:47.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 -- 172.21.15.6:0/3159259313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8101210 msgr2=0x7f95c8101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 --2- 172.21.15.6:0/3159259313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8101210 0x7f95c8101630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009900 tx=0x7f95b802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 -- 172.21.15.6:0/3159259313 shutdown_connections 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 --2- 172.21.15.6:0/3159259313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8102dc0 0x7f95c8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 --2- 172.21.15.6:0/3159259313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8102400 0x7f95c8102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 --2- 172.21.15.6:0/3159259313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8101210 0x7f95c8101630 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 -- 172.21.15.6:0/3159259313 >> 172.21.15.6:0/3159259313 conn(0x7f95c80fc760 msgr2=0x7f95c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 -- 172.21.15.6:0/3159259313 shutdown_connections 2024-02-26T06:27:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.152+0000 7f95ce400640 1 -- 172.21.15.6:0/3159259313 wait complete. 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 Processor -- start 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 -- start start 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 0x7f95c8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8102400 0x7f95c81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8102dc0 0x7f95c81a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81a0f60 con 0x7f95c8102dc0 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81a10d0 con 0x7f95c8101210 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.153+0000 7f95ce400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81a1290 con 0x7f95c8102400 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 0x7f95c8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 0x7f95c8199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55272/0 (socket says 172.21.15.6:55272) 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 -- 172.21.15.6:0/1686560186 learned_addr learned my addr 172.21.15.6:0/1686560186 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7e00640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8102dc0 0x7f95c81a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c6a00640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8102400 0x7f95c81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 -- 172.21.15.6:0/1686560186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8102400 msgr2=0x7f95c81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8102400 0x7f95c81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 -- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8102dc0 msgr2=0x7f95c81a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8102dc0 0x7f95c81a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.154+0000 7f95c7400640 1 -- 172.21.15.6:0/1686560186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0009840 con 0x7f95c8101210 2024-02-26T06:27:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.155+0000 7f95c7400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 0x7f95c8199470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005ab0 tx=0x7f95b8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.155+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80043d0 con 0x7f95c8101210 2024-02-26T06:27:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.155+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b8009560 con 0x7f95c8101210 2024-02-26T06:27:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.155+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95b80052d0 con 0x7f95c8101210 2024-02-26T06:27:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.155+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8003cd0 con 0x7f95c8101210 2024-02-26T06:27:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.155+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c81a18e0 con 0x7f95c8101210 2024-02-26T06:27:47.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.156+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c8068450 con 0x7f95c8101210 2024-02-26T06:27:47.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.158+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f95b8005440 con 0x7f95c8101210 2024-02-26T06:27:47.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.160+0000 7f95bfe00640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95ac0a20c0 0x7f95ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.160+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f95b80f0500 con 0x7f95c8101210 2024-02-26T06:27:47.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.160+0000 7f95c6a00640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95ac0a20c0 0x7f95ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.161+0000 7f95c6a00640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95ac0a20c0 0x7f95ac0a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f95c819a980 tx=0x7f95b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:47.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.162+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95b80b9460 con 0x7f95c8101210 2024-02-26T06:27:47.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:46.836+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:27:47.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:46.836+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:27:47.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:46.836+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:27:47.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:46.836+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:27:47.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:46.836+0000 7f989b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:27:47.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.358+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95c810ce00 con 0x7f95c8101210 2024-02-26T06:27:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.360+0000 7f95bfe00640 1 -- 172.21.15.6:0/1686560186 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f95b80bcb20 con 0x7f95c8101210 2024-02-26T06:27:47.361 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:47.361 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:47.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95ac0a20c0 msgr2=0x7f95ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95ac0a20c0 0x7f95ac0a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f95c819a980 tx=0x7f95b4005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:47.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 msgr2=0x7f95c8199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 0x7f95c8199470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005ab0 tx=0x7f95b8005e50 comp rx=0 tx=0).stop 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 shutdown_connections 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95c8102dc0 0x7f95c81a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8102400 0x7f95c81999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f95ac0a20c0 0x7f95ac0a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 --2- 172.21.15.6:0/1686560186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95c8101210 0x7f95c8199470 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 >> 172.21.15.6:0/1686560186 conn(0x7f95c80fc760 msgr2=0x7f95c80733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 shutdown_connections 2024-02-26T06:27:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.363+0000 7f95ce400640 1 -- 172.21.15.6:0/1686560186 wait complete. 2024-02-26T06:27:47.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:27:47.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 -- 172.21.15.6:0/813050803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64106cb0 msgr2=0x7f1e64109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 --2- 172.21.15.6:0/813050803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64106cb0 0x7f1e64109100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009900 tx=0x7f1e5c02bc50 comp rx=0 tx=0).stop 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 -- 172.21.15.6:0/813050803 shutdown_connections 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 --2- 172.21.15.6:0/813050803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64106cb0 0x7f1e64109100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 --2- 172.21.15.6:0/813050803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64104380 0x7f1e64106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 --2- 172.21.15.6:0/813050803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64101a50 0x7f1e64103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 -- 172.21.15.6:0/813050803 >> 172.21.15.6:0/813050803 conn(0x7f1e640fb460 msgr2=0x7f1e640fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 -- 172.21.15.6:0/813050803 shutdown_connections 2024-02-26T06:27:47.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.596+0000 7f1e63e00640 1 -- 172.21.15.6:0/813050803 wait complete. 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 Processor -- start 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 -- start start 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 0x7f1e64196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64104380 0x7f1e64197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64106cb0 0x7f1e6419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419eac0 con 0x7f1e64101a50 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419ec30 con 0x7f1e64104380 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e63e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419edf0 con 0x7f1e64106cb0 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.598+0000 7f1e62a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 0x7f1e64196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64104380 0x7f1e64197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 0x7f1e64196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40888/0 (socket says 172.21.15.6:40888) 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62a00640 1 -- 172.21.15.6:0/919366863 learned_addr learned my addr 172.21.15.6:0/919366863 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e63400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64106cb0 0x7f1e6419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 -- 172.21.15.6:0/919366863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64106cb0 msgr2=0x7f1e6419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64106cb0 0x7f1e6419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 -- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 msgr2=0x7f1e64196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 0x7f1e64196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 -- 172.21.15.6:0/919366863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54009490 con 0x7f1e64104380 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62a00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 0x7f1e64196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e63400640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64106cb0 0x7f1e6419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.599+0000 7f1e62000640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64104380 0x7f1e64197510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c005fd0 tx=0x7f1e4c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.600+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c003b70 con 0x7f1e64104380 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.600+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c003cd0 con 0x7f1e64104380 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.600+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c009840 con 0x7f1e64104380 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.600+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c010890 con 0x7f1e64104380 2024-02-26T06:27:47.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.600+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c009560 con 0x7f1e64104380 2024-02-26T06:27:47.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.601+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e640fcf50 con 0x7f1e64104380 2024-02-26T06:27:47.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.603+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1e4c003e40 con 0x7f1e64104380 2024-02-26T06:27:47.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.604+0000 7f1e68a00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1e440a20c0 0x7f1e440a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:47.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.604+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1e4c014070 con 0x7f1e64104380 2024-02-26T06:27:47.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.604+0000 7f1e62a00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1e440a20c0 0x7f1e440a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:47.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.605+0000 7f1e62a00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1e440a20c0 0x7f1e440a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e54011690 tx=0x7f1e54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:47.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.608+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e4c0971b0 con 0x7f1e64104380 2024-02-26T06:27:47.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:47 smithi090 ceph-mon[35681]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:27:47.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:47 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2689993171' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-26T06:27:47.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:47 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:27:47.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:47 smithi090 ceph-mon[35681]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:47 smithi006 ceph-mon[31790]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:27:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:47 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2689993171' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-26T06:27:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:47 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:27:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:47 smithi006 ceph-mon[31790]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:47.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:47 smithi167 ceph-mon[36229]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:27:47.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:47 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2689993171' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-26T06:27:47.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:47 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:27:47.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:47 smithi167 ceph-mon[36229]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:27:47.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.807+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e6402cdd0 con 0x7f1e64104380 2024-02-26T06:27:47.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.808+0000 7f1e68a00640 1 -- 172.21.15.6:0/919366863 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e4c09a870 con 0x7f1e64104380 2024-02-26T06:27:47.808 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1e440a20c0 msgr2=0x7f1e440a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1e440a20c0 0x7f1e440a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e54011690 tx=0x7f1e54005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64104380 msgr2=0x7f1e64197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64104380 0x7f1e64197510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c005fd0 tx=0x7f1e4c00c6a0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 shutdown_connections 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e64106cb0 0x7f1e6419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1e440a20c0 0x7f1e440a4580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e64104380 0x7f1e64197510 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.810+0000 7f1e63e00640 1 --2- 172.21.15.6:0/919366863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e64101a50 0x7f1e64196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:47.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.811+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 >> 172.21.15.6:0/919366863 conn(0x7f1e640fb460 msgr2=0x7f1e64102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:47.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.811+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 shutdown_connections 2024-02-26T06:27:47.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:47.811+0000 7f1e63e00640 1 -- 172.21.15.6:0/919366863 wait complete. 2024-02-26T06:27:47.830 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 -- 172.21.15.6:0/373320727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac104380 msgr2=0x7fd6ac106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 --2- 172.21.15.6:0/373320727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac104380 0x7fd6ac106770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd698009860 tx=0x7fd69802bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 -- 172.21.15.6:0/373320727 shutdown_connections 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 --2- 172.21.15.6:0/373320727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac106cb0 0x7fd6ac109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 --2- 172.21.15.6:0/373320727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac104380 0x7fd6ac106770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 --2- 172.21.15.6:0/373320727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac101a50 0x7fd6ac103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.056+0000 7fd6b2800640 1 -- 172.21.15.6:0/373320727 >> 172.21.15.6:0/373320727 conn(0x7fd6ac0fb460 msgr2=0x7fd6ac0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:48.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.057+0000 7fd6b2800640 1 -- 172.21.15.6:0/373320727 shutdown_connections 2024-02-26T06:27:48.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.057+0000 7fd6b2800640 1 -- 172.21.15.6:0/373320727 wait complete. 2024-02-26T06:27:48.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 Processor -- start 2024-02-26T06:27:48.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 -- start start 2024-02-26T06:27:48.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101a50 0x7fd6ac1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 0x7fd6ac0ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac106cb0 0x7fd6ac0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1006a0 con 0x7fd6ac104380 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac100810 con 0x7fd6ac101a50 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a10c0 con 0x7fd6ac106cb0 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 0x7fd6ac0ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 0x7fd6ac0ffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40902/0 (socket says 172.21.15.6:40902) 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.058+0000 7fd6b0a00640 1 -- 172.21.15.6:0/3641506053 learned_addr learned my addr 172.21.15.6:0/3641506053 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b1e00640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac106cb0 0x7fd6ac0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b1400640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101a50 0x7fd6ac1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b0a00640 1 -- 172.21.15.6:0/3641506053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac106cb0 msgr2=0x7fd6ac0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b0a00640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac106cb0 0x7fd6ac0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b0a00640 1 -- 172.21.15.6:0/3641506053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101a50 msgr2=0x7fd6ac1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b0a00640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101a50 0x7fd6ac1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b0a00640 1 -- 172.21.15.6:0/3641506053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009490 con 0x7fd6ac104380 2024-02-26T06:27:48.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.059+0000 7fd6b0a00640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 0x7fd6ac0ffa00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd698009780 tx=0x7fd698005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:48.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.060+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69802c510 con 0x7fd6ac104380 2024-02-26T06:27:48.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.060+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac1a1310 con 0x7fd6ac104380 2024-02-26T06:27:48.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.060+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69802cb50 con 0x7fd6ac104380 2024-02-26T06:27:48.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.060+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698003850 con 0x7fd6ac104380 2024-02-26T06:27:48.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.060+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1a1680 con 0x7fd6ac104380 2024-02-26T06:27:48.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.061+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac0fcf50 con 0x7fd6ac104380 2024-02-26T06:27:48.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.063+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd69803b040 con 0x7fd6ac104380 2024-02-26T06:27:48.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.064+0000 7fd6aa000640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd6940a20c0 0x7fd6940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.064+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd6980ef890 con 0x7fd6ac104380 2024-02-26T06:27:48.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.065+0000 7fd6b1400640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd6940a20c0 0x7fd6940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.065+0000 7fd6b1400640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd6940a20c0 0x7fd6940a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0005fd0 tx=0x7fd6a0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:48.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.069+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6980b88a0 con 0x7fd6ac104380 2024-02-26T06:27:48.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.271+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6ac02cdd0 con 0x7fd6ac104380 2024-02-26T06:27:48.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.272+0000 7fd6aa000640 1 -- 172.21.15.6:0/3641506053 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6980b8030 con 0x7fd6ac104380 2024-02-26T06:27:48.273 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:27:48.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd6940a20c0 msgr2=0x7fd6940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd6940a20c0 0x7fd6940a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0005fd0 tx=0x7fd6a0010040 comp rx=0 tx=0).stop 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 msgr2=0x7fd6ac0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 0x7fd6ac0ffa00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd698009780 tx=0x7fd698005d50 comp rx=0 tx=0).stop 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 shutdown_connections 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac106cb0 0x7fd6ac0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac104380 0x7fd6ac0ffa00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fd6940a20c0 0x7fd6940a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 --2- 172.21.15.6:0/3641506053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101a50 0x7fd6ac1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 >> 172.21.15.6:0/3641506053 conn(0x7fd6ac0fb460 msgr2=0x7fd6ac1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 shutdown_connections 2024-02-26T06:27:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.275+0000 7fd6b2800640 1 -- 172.21.15.6:0/3641506053 wait complete. 2024-02-26T06:27:48.296 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:27:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 -- 172.21.15.6:0/2961120448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501043b0 msgr2=0x7f7250104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 --2- 172.21.15.6:0/2961120448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501043b0 0x7f7250104830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f723c009900 tx=0x7f723c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 -- 172.21.15.6:0/2961120448 shutdown_connections 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 --2- 172.21.15.6:0/2961120448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7250104d70 0x7f725010b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 --2- 172.21.15.6:0/2961120448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501043b0 0x7f7250104830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 --2- 172.21.15.6:0/2961120448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501031c0 0x7f72501035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 -- 172.21.15.6:0/2961120448 >> 172.21.15.6:0/2961120448 conn(0x7f72500fe750 msgr2=0x7f7250100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 -- 172.21.15.6:0/2961120448 shutdown_connections 2024-02-26T06:27:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.527+0000 7f7254a00640 1 -- 172.21.15.6:0/2961120448 wait complete. 2024-02-26T06:27:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.528+0000 7f7254a00640 1 Processor -- start 2024-02-26T06:27:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 -- start start 2024-02-26T06:27:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501031c0 0x7f7250199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 0x7f7250199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7250104d70 0x7f72501a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501a0d20 con 0x7f72501031c0 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501a0e90 con 0x7f72501043b0 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f7254a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501a1050 con 0x7f7250104d70 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f724de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 0x7f7250199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f724de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 0x7f7250199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55324/0 (socket says 172.21.15.6:55324) 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f724de00640 1 -- 172.21.15.6:0/2414842269 learned_addr learned my addr 172.21.15.6:0/2414842269 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f724e800640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501031c0 0x7f7250199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.529+0000 7f724f200640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7250104d70 0x7f72501a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f724de00640 1 -- 172.21.15.6:0/2414842269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7250104d70 msgr2=0x7f72501a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f724de00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7250104d70 0x7f72501a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f724de00640 1 -- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501031c0 msgr2=0x7f7250199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f724de00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501031c0 0x7f7250199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f724de00640 1 -- 172.21.15.6:0/2414842269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c009560 con 0x7f72501043b0 2024-02-26T06:27:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f724de00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 0x7f7250199770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f723c02c6b0 tx=0x7f723c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:48.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0044c0 con 0x7f72501043b0 2024-02-26T06:27:48.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.530+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72501a12d0 con 0x7f72501043b0 2024-02-26T06:27:48.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.531+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72501a1830 con 0x7f72501043b0 2024-02-26T06:27:48.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.531+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f723c0052d0 con 0x7f72501043b0 2024-02-26T06:27:48.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.531+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c003de0 con 0x7f72501043b0 2024-02-26T06:27:48.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.532+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72300052c0 con 0x7f72501043b0 2024-02-26T06:27:48.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.534+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f723c0035f0 con 0x7f72501043b0 2024-02-26T06:27:48.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.535+0000 7f7247400640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f72340a2150 0x7f72340a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.535+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f723c0efe50 con 0x7f72501043b0 2024-02-26T06:27:48.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.535+0000 7f724e800640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f72340a2150 0x7f72340a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.536+0000 7f724e800640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f72340a2150 0x7f72340a4610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7250104210 tx=0x7f7240005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.539+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f723c0b8d60 con 0x7f72501043b0 2024-02-26T06:27:48.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:48 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1686560186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:48.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:48 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/919366863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:48.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:48 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3641506053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:48.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:48 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1686560186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:48.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:48 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/919366863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:48.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:48 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3641506053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:48.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.743+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7230005840 con 0x7f72501043b0 2024-02-26T06:27:48.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.744+0000 7f7247400640 1 -- 172.21.15.6:0/2414842269 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f723c0bc420 con 0x7f72501043b0 2024-02-26T06:27:48.744 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:27:48.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:48 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1686560186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:48.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:48 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/919366863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:48.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:48 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3641506053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f72340a2150 msgr2=0x7f72340a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f72340a2150 0x7f72340a4610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7250104210 tx=0x7f7240005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 msgr2=0x7f7250199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 0x7f7250199770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f723c02c6b0 tx=0x7f723c0040c0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 shutdown_connections 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7250104d70 0x7f72501a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f72340a2150 0x7f72340a4610 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72501043b0 0x7f7250199770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 --2- 172.21.15.6:0/2414842269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72501031c0 0x7f7250199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.746+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 >> 172.21.15.6:0/2414842269 conn(0x7f72500fe750 msgr2=0x7f72501076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.747+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 shutdown_connections 2024-02-26T06:27:48.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.747+0000 7f7254a00640 1 -- 172.21.15.6:0/2414842269 wait complete. 2024-02-26T06:27:48.765 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:48.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 -- 172.21.15.6:0/2957218447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 msgr2=0x7f74f0074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2957218447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0074060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009860 tx=0x7f74dc02bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 -- 172.21.15.6:0/2957218447 shutdown_connections 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2957218447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f0108f60 0x7f74f010b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2957218447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0074060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2957218447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f00751c0 0x7f74f0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 -- 172.21.15.6:0/2957218447 >> 172.21.15.6:0/2957218447 conn(0x7f74f00fc610 msgr2=0x7f74f00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 -- 172.21.15.6:0/2957218447 shutdown_connections 2024-02-26T06:27:48.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.961+0000 7f74f4c00640 1 -- 172.21.15.6:0/2957218447 wait complete. 2024-02-26T06:27:48.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 Processor -- start 2024-02-26T06:27:48.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 -- start start 2024-02-26T06:27:48.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f00751c0 0x7f74f0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0108f60 0x7f74f01a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f01a0df0 con 0x7f74f0108f60 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f01a0f60 con 0x7f74f0073be0 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.962+0000 7f74f4c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f01a1120 con 0x7f74f00751c0 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55334/0 (socket says 172.21.15.6:55334) 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 -- 172.21.15.6:0/2008584722 learned_addr learned my addr 172.21.15.6:0/2008584722 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74eea00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f00751c0 0x7f74f0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74efe00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0108f60 0x7f74f01a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 -- 172.21.15.6:0/2008584722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f00751c0 msgr2=0x7f74f0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f00751c0 0x7f74f0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 -- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0108f60 msgr2=0x7f74f01a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0108f60 0x7f74f01a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 -- 172.21.15.6:0/2008584722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0009560 con 0x7f74f0073be0 2024-02-26T06:27:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.963+0000 7f74ef400640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0199350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74e000e9c0 tx=0x7f74e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.964+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000cde0 con 0x7f74f0073be0 2024-02-26T06:27:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.964+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e00035f0 con 0x7f74f0073be0 2024-02-26T06:27:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.964+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc009490 con 0x7f74f0073be0 2024-02-26T06:27:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.964+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0010670 con 0x7f74f0073be0 2024-02-26T06:27:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.964+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f01a17a0 con 0x7f74f0073be0 2024-02-26T06:27:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.965+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f0107c90 con 0x7f74f0073be0 2024-02-26T06:27:48.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.967+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f74e0003ad0 con 0x7f74f0073be0 2024-02-26T06:27:48.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.968+0000 7f74e7e00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f74d40a21c0 0x7f74d40a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:48.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.968+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f74e0014070 con 0x7f74f0073be0 2024-02-26T06:27:48.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.968+0000 7f74eea00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f74d40a21c0 0x7f74d40a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:48.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.969+0000 7f74eea00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f74d40a21c0 0x7f74d40a4680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f74dc02c180 tx=0x7f74dc034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:48.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:48.972+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74e0097120 con 0x7f74f0073be0 2024-02-26T06:27:49.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.178+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74f010eca0 con 0x7f74f0073be0 2024-02-26T06:27:49.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.181+0000 7f74e7e00640 1 -- 172.21.15.6:0/2008584722 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f74e0003760 con 0x7f74f0073be0 2024-02-26T06:27:49.181 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:49.182 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f74d40a21c0 msgr2=0x7f74d40a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f74d40a21c0 0x7f74d40a4680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f74dc02c180 tx=0x7f74dc034c00 comp rx=0 tx=0).stop 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 msgr2=0x7f74f0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0199350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74e000e9c0 tx=0x7f74e000ee80 comp rx=0 tx=0).stop 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 shutdown_connections 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0108f60 0x7f74f01a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f00751c0 0x7f74f0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f74d40a21c0 0x7f74d40a4680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.183+0000 7f74f4c00640 1 --2- 172.21.15.6:0/2008584722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f0073be0 0x7f74f0199350 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.184+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 >> 172.21.15.6:0/2008584722 conn(0x7f74f00fc610 msgr2=0x7f74f0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:49.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.184+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 shutdown_connections 2024-02-26T06:27:49.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.184+0000 7f74f4c00640 1 -- 172.21.15.6:0/2008584722 wait complete. 2024-02-26T06:27:49.203 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:27:49.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:27:49.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.408+0000 7ff413e00640 1 -- 172.21.15.6:0/3400945173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 msgr2=0x7ff414104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.408+0000 7ff413e00640 1 --2- 172.21.15.6:0/3400945173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff414104850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff400009900 tx=0x7ff40002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:49.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 -- 172.21.15.6:0/3400945173 shutdown_connections 2024-02-26T06:27:49.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 --2- 172.21.15.6:0/3400945173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff414104d90 0x7ff41410b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 --2- 172.21.15.6:0/3400945173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff414104850 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 --2- 172.21.15.6:0/3400945173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 0x7ff414103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 -- 172.21.15.6:0/3400945173 >> 172.21.15.6:0/3400945173 conn(0x7ff4140fe750 msgr2=0x7ff414100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:49.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 -- 172.21.15.6:0/3400945173 shutdown_connections 2024-02-26T06:27:49.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.409+0000 7ff413e00640 1 -- 172.21.15.6:0/3400945173 wait complete. 2024-02-26T06:27:49.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.410+0000 7ff413e00640 1 Processor -- start 2024-02-26T06:27:49.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.410+0000 7ff413e00640 1 -- start start 2024-02-26T06:27:49.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.410+0000 7ff413e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 0x7ff414199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.410+0000 7ff413e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff4141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.410+0000 7ff413e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff414104d90 0x7ff4141a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.410+0000 7ff413e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4141a0d70 con 0x7ff4141043d0 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 0x7ff414199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff4141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 0x7ff414199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:44662/0 (socket says 172.21.15.6:44662) 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412a00640 1 -- 172.21.15.6:0/1361147107 learned_addr learned my addr 172.21.15.6:0/1361147107 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff413400640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff414104d90 0x7ff4141a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4141a0ee0 con 0x7ff414104d90 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4141a10a0 con 0x7ff4141031e0 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412000640 1 -- 172.21.15.6:0/1361147107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 msgr2=0x7ff414199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412000640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 0x7ff414199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412000640 1 -- 172.21.15.6:0/1361147107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff414104d90 msgr2=0x7ff4141a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.411+0000 7ff412000640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff414104d90 0x7ff4141a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.412+0000 7ff412000640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c009840 con 0x7ff4141043d0 2024-02-26T06:27:49.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.412+0000 7ff412000640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff4141997c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff400000c00 tx=0x7ff400005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.412+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000043d0 con 0x7ff4141043d0 2024-02-26T06:27:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.412+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff400009560 con 0x7ff4141043d0 2024-02-26T06:27:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.412+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4000052d0 con 0x7ff4141043d0 2024-02-26T06:27:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.412+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400003cd0 con 0x7ff4141043d0 2024-02-26T06:27:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.413+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4141a16c0 con 0x7ff4141043d0 2024-02-26T06:27:49.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.414+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41404eee0 con 0x7ff4141043d0 2024-02-26T06:27:49.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.417+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7ff400005440 con 0x7ff4141043d0 2024-02-26T06:27:49.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.418+0000 7ff418a00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff3f80a2150 0x7ff3f80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.418+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff4000f03e0 con 0x7ff4141043d0 2024-02-26T06:27:49.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.419+0000 7ff412a00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff3f80a2150 0x7ff3f80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.419+0000 7ff412a00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff3f80a2150 0x7ff3f80a4610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff4040058c0 tx=0x7ff404005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:49.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.422+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4000b9350 con 0x7ff4141043d0 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2414842269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2008584722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:49.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:49 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:49.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.623+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff41410ebf0 con 0x7ff4141043d0 2024-02-26T06:27:49.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.624+0000 7ff418a00640 1 -- 172.21.15.6:0/1361147107 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff400043020 con 0x7ff4141043d0 2024-02-26T06:27:49.624 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff3f80a2150 msgr2=0x7ff3f80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff3f80a2150 0x7ff3f80a4610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff4040058c0 tx=0x7ff404005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 msgr2=0x7ff4141997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff4141997c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff400000c00 tx=0x7ff400005e50 comp rx=0 tx=0).stop 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 shutdown_connections 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff3f80a2150 0x7ff3f80a4610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff414104d90 0x7ff4141a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4141043d0 0x7ff4141997c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 --2- 172.21.15.6:0/1361147107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4141031e0 0x7ff414199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 >> 172.21.15.6:0/1361147107 conn(0x7ff4140fe750 msgr2=0x7ff414107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 shutdown_connections 2024-02-26T06:27:49.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.626+0000 7ff413e00640 1 -- 172.21.15.6:0/1361147107 wait complete. 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2414842269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2008584722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:49.641 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:49 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:49.643 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2414842269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2008584722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:49.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:49 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 -- 172.21.15.6:0/3243201182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1860102e00 msgr2=0x7f1860109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 --2- 172.21.15.6:0/3243201182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1860102e00 0x7f1860109690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1858009860 tx=0x7f185802bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 -- 172.21.15.6:0/3243201182 shutdown_connections 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 --2- 172.21.15.6:0/3243201182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1860102e00 0x7f1860109690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 --2- 172.21.15.6:0/3243201182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1860102440 0x7f18601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 --2- 172.21.15.6:0/3243201182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18600ff470 0x7f18600ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.827+0000 7f1868400640 1 -- 172.21.15.6:0/3243201182 >> 172.21.15.6:0/3243201182 conn(0x7f18600fada0 msgr2=0x7f18600fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.828+0000 7f1868400640 1 -- 172.21.15.6:0/3243201182 shutdown_connections 2024-02-26T06:27:49.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.828+0000 7f1868400640 1 -- 172.21.15.6:0/3243201182 wait complete. 2024-02-26T06:27:49.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.828+0000 7f1868400640 1 Processor -- start 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 -- start start 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 0x7f18601971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1860102440 0x7f1860197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1860102e00 0x7f186019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019ecc0 con 0x7f1860102440 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019ee30 con 0x7f18600ff470 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1868400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019eff0 con 0x7f1860102e00 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1865a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 0x7f18601971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1865a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 0x7f18601971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55380/0 (socket says 172.21.15.6:55380) 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1865000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1860102440 0x7f1860197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1865a00640 1 -- 172.21.15.6:0/1052155505 learned_addr learned my addr 172.21.15.6:0/1052155505 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.829+0000 7f1866400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1860102e00 0x7f186019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1865a00640 1 -- 172.21.15.6:0/1052155505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1860102e00 msgr2=0x7f186019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1865a00640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1860102e00 0x7f186019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1865a00640 1 -- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1860102440 msgr2=0x7f1860197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1865a00640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1860102440 0x7f1860197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1865a00640 1 -- 172.21.15.6:0/1052155505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c009560 con 0x7f18600ff470 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1865a00640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 0x7f18601971d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f185400eb70 tx=0x7f185400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854003b70 con 0x7f18600ff470 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f185400ee70 con 0x7f18600ff470 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540107f0 con 0x7f18600ff470 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1858009490 con 0x7f18600ff470 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.830+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186019f600 con 0x7f18600ff470 2024-02-26T06:27:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.831+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860108510 con 0x7f18600ff470 2024-02-26T06:27:49.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.833+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f1854004020 con 0x7f18600ff470 2024-02-26T06:27:49.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.834+0000 7f185e000640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a2150 0x7f18480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:49.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.834+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1854014070 con 0x7f18600ff470 2024-02-26T06:27:49.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.835+0000 7f1865000640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a2150 0x7f18480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:49.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.835+0000 7f1865000640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a2150 0x7f18480a4610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f18601986e0 tx=0x7f184c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:49.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:49.838+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18540972b0 con 0x7f18600ff470 2024-02-26T06:27:50.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.035+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f186010cbd0 con 0x7f18600ff470 2024-02-26T06:27:50.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.035+0000 7f185e000640 1 -- 172.21.15.6:0/1052155505 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f185409a970 con 0x7f18600ff470 2024-02-26T06:27:50.036 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a2150 msgr2=0x7f18480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a2150 0x7f18480a4610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f18601986e0 tx=0x7f184c009260 comp rx=0 tx=0).stop 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 msgr2=0x7f18601971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 0x7f18601971d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f185400eb70 tx=0x7f185400c6a0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 shutdown_connections 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1860102e00 0x7f186019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1860102440 0x7f1860197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f18480a2150 0x7f18480a4610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 --2- 172.21.15.6:0/1052155505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18600ff470 0x7f18601971d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 >> 172.21.15.6:0/1052155505 conn(0x7f18600fada0 msgr2=0x7f18601056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:50.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.038+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 shutdown_connections 2024-02-26T06:27:50.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.039+0000 7f1868400640 1 -- 172.21.15.6:0/1052155505 wait complete. 2024-02-26T06:27:50.055 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.282+0000 7f5c75c00640 1 -- 172.21.15.6:0/3937726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70100fd0 msgr2=0x7f5c701013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.282+0000 7f5c75c00640 1 --2- 172.21.15.6:0/3937726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70100fd0 0x7f5c701013f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009900 tx=0x7f5c6002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.282+0000 7f5c75c00640 1 -- 172.21.15.6:0/3937726464 shutdown_connections 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.282+0000 7f5c75c00640 1 --2- 172.21.15.6:0/3937726464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c70102b80 0x7f5c70109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.282+0000 7f5c75c00640 1 --2- 172.21.15.6:0/3937726464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c701021c0 0x7f5c70102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.283+0000 7f5c75c00640 1 --2- 172.21.15.6:0/3937726464 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70100fd0 0x7f5c701013f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.283+0000 7f5c75c00640 1 -- 172.21.15.6:0/3937726464 >> 172.21.15.6:0/3937726464 conn(0x7f5c700fc540 msgr2=0x7f5c700fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:50.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.283+0000 7f5c75c00640 1 -- 172.21.15.6:0/3937726464 shutdown_connections 2024-02-26T06:27:50.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.283+0000 7f5c75c00640 1 -- 172.21.15.6:0/3937726464 wait complete. 2024-02-26T06:27:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.283+0000 7f5c75c00640 1 Processor -- start 2024-02-26T06:27:50.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 -- start start 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c70100fd0 0x7f5c70199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 0x7f5c701997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 0x7f5c701a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701a0d80 con 0x7f5c70102b80 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701a0ef0 con 0x7f5c701021c0 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701a10b0 con 0x7f5c70100fd0 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 0x7f5c701a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c6f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 0x7f5c701997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 0x7f5c701a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40990/0 (socket says 172.21.15.6:40990) 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c6f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 0x7f5c701997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55398/0 (socket says 172.21.15.6:55398) 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c75200640 1 -- 172.21.15.6:0/1029428601 learned_addr learned my addr 172.21.15.6:0/1029428601 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.284+0000 7f5c6fe00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c70100fd0 0x7f5c70199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6f400640 1 -- 172.21.15.6:0/1029428601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c70100fd0 msgr2=0x7f5c70199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6f400640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c70100fd0 0x7f5c70199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6f400640 1 -- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 msgr2=0x7f5c701a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6f400640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 0x7f5c701a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6f400640 1 -- 172.21.15.6:0/1029428601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009560 con 0x7f5c701021c0 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c75200640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 0x7f5c701a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6f400640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 0x7f5c701997d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00e9f0 tx=0x7f5c5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c00ccc0 con 0x7f5c701021c0 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c701a1390 con 0x7f5c701021c0 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c5c00ce20 con 0x7f5c701021c0 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c004a00 con 0x7f5c701021c0 2024-02-26T06:27:50.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.285+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c701a18a0 con 0x7f5c701021c0 2024-02-26T06:27:50.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.286+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7010ca10 con 0x7f5c701021c0 2024-02-26T06:27:50.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.289+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f5c5c0050b0 con 0x7f5c701021c0 2024-02-26T06:27:50.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.290+0000 7f5c6cc00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5c540a2150 0x7f5c540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.291+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5c5c014070 con 0x7f5c701021c0 2024-02-26T06:27:50.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.291+0000 7f5c6fe00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5c540a2150 0x7f5c540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.292+0000 7f5c6fe00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5c540a2150 0x7f5c540a4610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c6002c240 tx=0x7f5c6002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:50.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.294+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c5c096c30 con 0x7f5c701021c0 2024-02-26T06:27:50.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.514+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c701a1d30 con 0x7f5c701021c0 2024-02-26T06:27:50.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.515+0000 7f5c6cc00640 1 -- 172.21.15.6:0/1029428601 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c5c09a2f0 con 0x7f5c701021c0 2024-02-26T06:27:50.515 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5c540a2150 msgr2=0x7f5c540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5c540a2150 0x7f5c540a4610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c6002c240 tx=0x7f5c6002c7e0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 msgr2=0x7f5c701997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 0x7f5c701997d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00e9f0 tx=0x7f5c5c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 shutdown_connections 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c70102b80 0x7f5c701a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f5c540a2150 0x7f5c540a4610 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c701021c0 0x7f5c701997d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.517+0000 7f5c75c00640 1 --2- 172.21.15.6:0/1029428601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c70100fd0 0x7f5c70199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.518+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 >> 172.21.15.6:0/1029428601 conn(0x7f5c700fc540 msgr2=0x7f5c700733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:50.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.518+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 shutdown_connections 2024-02-26T06:27:50.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.518+0000 7f5c75c00640 1 -- 172.21.15.6:0/1029428601 wait complete. 2024-02-26T06:27:50.537 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-26T06:27:50.537 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 -- 172.21.15.6:0/472969743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a0073be0 msgr2=0x7f14a0074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 --2- 172.21.15.6:0/472969743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a0073be0 0x7f14a0074060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1488009860 tx=0x7f148802bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 -- 172.21.15.6:0/472969743 shutdown_connections 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 --2- 172.21.15.6:0/472969743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 0x7f14a010b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 --2- 172.21.15.6:0/472969743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a0073be0 0x7f14a0074060 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 --2- 172.21.15.6:0/472969743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a00751c0 0x7f14a0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 -- 172.21.15.6:0/472969743 >> 172.21.15.6:0/472969743 conn(0x7f14a00fc610 msgr2=0x7f14a00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 -- 172.21.15.6:0/472969743 shutdown_connections 2024-02-26T06:27:50.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.725+0000 7f149f200640 1 -- 172.21.15.6:0/472969743 wait complete. 2024-02-26T06:27:50.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.726+0000 7f149f200640 1 Processor -- start 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.726+0000 7f149f200640 1 -- start start 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.726+0000 7f149f200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a0073be0 0x7f14a0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 0x7f14a0199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 0x7f14a01a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149f200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01a0df0 con 0x7f14a0073be0 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01a0f60 con 0x7f14a0108f60 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01a1120 con 0x7f14a00751c0 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 0x7f14a0199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 0x7f14a0199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:44726/0 (socket says 172.21.15.6:44726) 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 -- 172.21.15.6:0/483501381 learned_addr learned my addr 172.21.15.6:0/483501381 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a0073be0 0x7f14a0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149e800640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 0x7f14a01a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 -- 172.21.15.6:0/483501381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 msgr2=0x7f14a01a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 0x7f14a01a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 -- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a0073be0 msgr2=0x7f14a0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a0073be0 0x7f14a0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f1495400640 1 -- 172.21.15.6:0/483501381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1488009490 con 0x7f14a00751c0 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.727+0000 7f149e800640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 0x7f14a01a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.728+0000 7f1495400640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 0x7f14a0199840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1488034b10 tx=0x7f14880045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.728+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488005020 con 0x7f14a00751c0 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.728+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f148802c510 con 0x7f14a00751c0 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.728+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a01a13a0 con 0x7f14a00751c0 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.728+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488004a60 con 0x7f14a00751c0 2024-02-26T06:27:50.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.728+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a01a1760 con 0x7f14a00751c0 2024-02-26T06:27:50.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.729+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a0107c90 con 0x7f14a00751c0 2024-02-26T06:27:50.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.731+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f148803b040 con 0x7f14a00751c0 2024-02-26T06:27:50.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.733+0000 7f1497400640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f14840a21c0 0x7f14840a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:50.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.733+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f14880ef6c0 con 0x7f14a00751c0 2024-02-26T06:27:50.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.733+0000 7f149de00640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f14840a21c0 0x7f14840a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:50.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.734+0000 7f149de00640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f14840a21c0 0x7f14840a4680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1490009bd0 tx=0x7f1490005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:50.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:50.736+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14880b85d0 con 0x7f14a00751c0 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1361147107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1052155505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:50 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1029428601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1361147107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1052155505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:50 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1029428601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1361147107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1052155505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:50.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:50 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1029428601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:51.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.004+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14a010eef0 con 0x7f14a00751c0 2024-02-26T06:27:51.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.005+0000 7f1497400640 1 -- 172.21.15.6:0/483501381 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f14880bbc90 con 0x7f14a00751c0 2024-02-26T06:27:51.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:51.006 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:27:45.824981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:51.007 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-02-26T06:27:51.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f14840a21c0 msgr2=0x7f14840a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f14840a21c0 0x7f14840a4680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1490009bd0 tx=0x7f1490005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:51.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 msgr2=0x7f14a0199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 0x7f14a0199840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1488034b10 tx=0x7f14880045c0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 shutdown_connections 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f14840a21c0 0x7f14840a4680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14a0108f60 0x7f14a01a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a00751c0 0x7f14a0199840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 --2- 172.21.15.6:0/483501381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a0073be0 0x7f14a0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 >> 172.21.15.6:0/483501381 conn(0x7f14a00fc610 msgr2=0x7f14a0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 shutdown_connections 2024-02-26T06:27:51.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.008+0000 7f149f200640 1 -- 172.21.15.6:0/483501381 wait complete. 2024-02-26T06:27:51.027 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-26T06:27:51.027 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-26T06:27:51.027 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:27:51.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 -- 172.21.15.6:0/4205069602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150108f50 msgr2=0x7f615010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 --2- 172.21.15.6:0/4205069602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150108f50 0x7f615010b3e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6148009be0 tx=0x7f614802bf80 comp rx=0 tx=0).stop 2024-02-26T06:27:51.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 -- 172.21.15.6:0/4205069602 shutdown_connections 2024-02-26T06:27:51.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 --2- 172.21.15.6:0/4205069602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150108f50 0x7f615010b3e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 --2- 172.21.15.6:0/4205069602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150073b60 0x7f6150073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 --2- 172.21.15.6:0/4205069602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 0x7f6150073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.235+0000 7f6156000640 1 -- 172.21.15.6:0/4205069602 >> 172.21.15.6:0/4205069602 conn(0x7f61500fc610 msgr2=0x7f61500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:51.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.236+0000 7f6156000640 1 -- 172.21.15.6:0/4205069602 shutdown_connections 2024-02-26T06:27:51.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.236+0000 7f6156000640 1 -- 172.21.15.6:0/4205069602 wait complete. 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.236+0000 7f6156000640 1 Processor -- start 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 -- start start 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 0x7f6150199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 0x7f61501997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150108f50 0x7f61501a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61501a0d70 con 0x7f6150075140 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61501a0ee0 con 0x7f6150073b60 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6156000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61501a10a0 con 0x7f6150108f50 2024-02-26T06:27:51.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6154c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 0x7f6150199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6154c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 0x7f6150199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55430/0 (socket says 172.21.15.6:55430) 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6154c00640 1 -- 172.21.15.6:0/343794091 learned_addr learned my addr 172.21.15.6:0/343794091 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f614fe00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 0x7f61501997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.237+0000 7f6155600640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150108f50 0x7f61501a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6154c00640 1 -- 172.21.15.6:0/343794091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150108f50 msgr2=0x7f61501a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6154c00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150108f50 0x7f61501a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6154c00640 1 -- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 msgr2=0x7f61501997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6154c00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 0x7f61501997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6154c00640 1 -- 172.21.15.6:0/343794091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148009840 con 0x7f6150073b60 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6155600640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150108f50 0x7f61501a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f614fe00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 0x7f61501997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:51.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6154c00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 0x7f6150199280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9f0 tx=0x7f614400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144009700 con 0x7f6150073b60 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f614400cd80 con 0x7f6150073b60 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144018920 con 0x7f6150073b60 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61501a1380 con 0x7f6150073b60 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.238+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61501a18e0 con 0x7f6150073b60 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.240+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6150107d10 con 0x7f6150073b60 2024-02-26T06:27:51.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.242+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f6144018a80 con 0x7f6150073b60 2024-02-26T06:27:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.243+0000 7f614d600640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f61380a21c0 0x7f61380a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.243+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f6144014070 con 0x7f6150073b60 2024-02-26T06:27:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.244+0000 7f614fe00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f61380a21c0 0x7f61380a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.244+0000 7f614fe00640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f61380a21c0 0x7f61380a4680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f615019a790 tx=0x7f613c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:51.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.247+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6144052d20 con 0x7f6150073b60 2024-02-26T06:27:51.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.465+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f615002ce40 con 0x7f61380a21c0 2024-02-26T06:27:51.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.478+0000 7f614d600640 1 -- 172.21.15.6:0/343794091 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343101 (secure 0 0 0) 0x7f615002ce40 con 0x7f61380a21c0 2024-02-26T06:27:51.485 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:51.487 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:27:51.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.485+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f61380a21c0 msgr2=0x7f61380a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.485+0000 7f6156000640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f61380a21c0 0x7f61380a4680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f615019a790 tx=0x7f613c009260 comp rx=0 tx=0).stop 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.485+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 msgr2=0x7f6150199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.485+0000 7f6156000640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 0x7f6150199280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9f0 tx=0x7f614400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 shutdown_connections 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150108f50 0x7f61501a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6150075140 0x7f61501997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f61380a21c0 0x7f61380a4680 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 --2- 172.21.15.6:0/343794091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6150073b60 0x7f6150199280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 >> 172.21.15.6:0/343794091 conn(0x7f61500fc610 msgr2=0x7f6150104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 shutdown_connections 2024-02-26T06:27:51.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.486+0000 7f6156000640 1 -- 172.21.15.6:0/343794091 wait complete. 2024-02-26T06:27:51.504 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-02-26T06:27:50.170805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664603,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1616,"num_read_kb":942,"num_write":441,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181457,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383368,"kb_used_data":13416,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558456,"statfs":{"total":1151940427776,"available":1151547858944,"internally_reserved":0,"allocated":13737984,"data_stored":7456558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6775,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":79,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005615"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125767+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:26:42.125767+0000","last_peered":"2024-02-26T06:26:42.125767+0000","last_clean":"2024-02-26T06:26:42.125767+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:26:42.125767+0000","last_undegraded":"2024-02-26T06:26:42.125767+0000","last_fullsized":"2024-02-26T06:26:42.125767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121615+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:26:42.121615+0000","last_peered":"2024-02-26T06:26:42.121615+0000","last_clean":"2024-02-26T06:26:42.121615+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:26:42.121615+0000","last_undegraded":"2024-02-26T06:26:42.121615+0000","last_fullsized":"2024-02-26T06:26:42.121615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128284+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:26:42.128284+0000","last_peered":"2024-02-26T06:26:42.128284+0000","last_clean":"2024-02-26T06:26:42.128284+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:26:42.128284+0000","last_undegraded":"2024-02-26T06:26:42.128284+0000","last_fullsized":"2024-02-26T06:26:42.128284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106994+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:26:42.106994+0000","last_peered":"2024-02-26T06:26:42.106994+0000","last_clean":"2024-02-26T06:26:42.106994+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:26:42.106994+0000","last_undegraded":"2024-02-26T06:26:42.106994+0000","last_fullsized":"2024-02-26T06:26:42.106994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.102169+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:26:42.102169+0000","last_peered":"2024-02-26T06:26:42.102169+0000","last_clean":"2024-02-26T06:26:42.102169+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:26:42.102169+0000","last_undegraded":"2024-02-26T06:26:42.102169+0000","last_fullsized":"2024-02-26T06:26:42.102169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105559+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:26:42.105559+0000","last_peered":"2024-02-26T06:26:42.105559+0000","last_clean":"2024-02-26T06:26:42.105559+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:26:42.105559+0000","last_undegraded":"2024-02-26T06:26:42.105559+0000","last_fullsized":"2024-02-26T06:26:42.105559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114163+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:26:42.114163+0000","last_peered":"2024-02-26T06:26:42.114163+0000","last_clean":"2024-02-26T06:26:42.114163+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:26:42.114163+0000","last_undegraded":"2024-02-26T06:26:42.114163+0000","last_fullsized":"2024-02-26T06:26:42.114163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131277+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:26:42.131277+0000","last_peered":"2024-02-26T06:26:42.131277+0000","last_clean":"2024-02-26T06:26:42.131277+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:26:42.131277+0000","last_undegraded":"2024-02-26T06:26:42.131277+0000","last_fullsized":"2024-02-26T06:26:42.131277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114350+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:26:42.114350+0000","last_peered":"2024-02-26T06:26:42.114350+0000","last_clean":"2024-02-26T06:26:42.114350+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:26:42.114350+0000","last_undegraded":"2024-02-26T06:26:42.114350+0000","last_fullsized":"2024-02-26T06:26:42.114350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107249+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:26:42.107249+0000","last_peered":"2024-02-26T06:26:42.107249+0000","last_clean":"2024-02-26T06:26:42.107249+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:26:42.107249+0000","last_undegraded":"2024-02-26T06:26:42.107249+0000","last_fullsized":"2024-02-26T06:26:42.107249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107797+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:26:42.107797+0000","last_peered":"2024-02-26T06:26:42.107797+0000","last_clean":"2024-02-26T06:26:42.107797+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:26:42.107797+0000","last_undegraded":"2024-02-26T06:26:42.107797+0000","last_fullsized":"2024-02-26T06:26:42.107797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100577+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:26:42.100577+0000","last_peered":"2024-02-26T06:26:42.100577+0000","last_clean":"2024-02-26T06:26:42.100577+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:26:42.100577+0000","last_undegraded":"2024-02-26T06:26:42.100577+0000","last_fullsized":"2024-02-26T06:26:42.100577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123475+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:26:42.123475+0000","last_peered":"2024-02-26T06:26:42.123475+0000","last_clean":"2024-02-26T06:26:42.123475+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:26:42.123475+0000","last_undegraded":"2024-02-26T06:26:42.123475+0000","last_fullsized":"2024-02-26T06:26:42.123475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127750+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:26:42.127750+0000","last_peered":"2024-02-26T06:26:42.127750+0000","last_clean":"2024-02-26T06:26:42.127750+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:26:42.127750+0000","last_undegraded":"2024-02-26T06:26:42.127750+0000","last_fullsized":"2024-02-26T06:26:42.127750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124332+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:26:42.124332+0000","last_peered":"2024-02-26T06:26:42.124332+0000","last_clean":"2024-02-26T06:26:42.124332+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:26:42.124332+0000","last_undegraded":"2024-02-26T06:26:42.124332+0000","last_fullsized":"2024-02-26T06:26:42.124332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129391+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:26:42.129391+0000","last_peered":"2024-02-26T06:26:42.129391+0000","last_clean":"2024-02-26T06:26:42.129391+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:26:42.129391+0000","last_undegraded":"2024-02-26T06:26:42.129391+0000","last_fullsized":"2024-02-26T06:26:42.129391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125804+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:26:42.125804+0000","last_peered":"2024-02-26T06:26:42.125804+0000","last_clean":"2024-02-26T06:26:42.125804+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:26:42.125804+0000","last_undegraded":"2024-02-26T06:26:42.125804+0000","last_fullsized":"2024-02-26T06:26:42.125804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.109782+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:26:42.109782+0000","last_peered":"2024-02-26T06:26:42.109782+0000","last_clean":"2024-02-26T06:26:42.109782+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:26:42.109782+0000","last_undegraded":"2024-02-26T06:26:42.109782+0000","last_fullsized":"2024-02-26T06:26:42.109782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125127+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:26:42.125127+0000","last_peered":"2024-02-26T06:26:42.125127+0000","last_clean":"2024-02-26T06:26:42.125127+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:26:42.125127+0000","last_undegraded":"2024-02-26T06:26:42.125127+0000","last_fullsized":"2024-02-26T06:26:42.125127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":179,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:48.114147+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:27:48.114147+0000","last_peered":"2024-02-26T06:27:48.114147+0000","last_clean":"2024-02-26T06:27:48.114147+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:27:48.114147+0000","last_undegraded":"2024-02-26T06:27:48.114147+0000","last_fullsized":"2024-02-26T06:27:48.114147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120848+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:26:42.120848+0000","last_peered":"2024-02-26T06:26:42.120848+0000","last_clean":"2024-02-26T06:26:42.120848+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:26:42.120848+0000","last_undegraded":"2024-02-26T06:26:42.120848+0000","last_fullsized":"2024-02-26T06:26:42.120848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869485+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:26:13.869485+0000","last_peered":"2024-02-26T06:26:13.869485+0000","last_clean":"2024-02-26T06:26:13.869485+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:26:13.869485+0000","last_undegraded":"2024-02-26T06:26:13.869485+0000","last_fullsized":"2024-02-26T06:26:13.869485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128676+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:26:42.128676+0000","last_peered":"2024-02-26T06:26:42.128676+0000","last_clean":"2024-02-26T06:26:42.128676+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:26:42.128676+0000","last_undegraded":"2024-02-26T06:26:42.128676+0000","last_fullsized":"2024-02-26T06:26:42.128676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.864796+0000","last_change":"2024-02-26T06:26:13.864219+0000","last_active":"2024-02-26T06:26:13.864796+0000","last_peered":"2024-02-26T06:26:13.864796+0000","last_clean":"2024-02-26T06:26:13.864796+0000","last_became_active":"2024-02-26T06:26:13.851415+0000","last_became_peered":"2024-02-26T06:26:13.851415+0000","last_unstale":"2024-02-26T06:26:13.864796+0000","last_undegraded":"2024-02-26T06:26:13.864796+0000","last_fullsized":"2024-02-26T06:26:13.864796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:46:36.684794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125570+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:26:42.125570+0000","last_peered":"2024-02-26T06:26:42.125570+0000","last_clean":"2024-02-26T06:26:42.125570+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:26:42.125570+0000","last_undegraded":"2024-02-26T06:26:42.125570+0000","last_fullsized":"2024-02-26T06:26:42.125570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.874338+0000","last_change":"2024-02-26T06:26:13.873833+0000","last_active":"2024-02-26T06:26:13.874338+0000","last_peered":"2024-02-26T06:26:13.874338+0000","last_clean":"2024-02-26T06:26:13.874338+0000","last_became_active":"2024-02-26T06:26:13.852310+0000","last_became_peered":"2024-02-26T06:26:13.852310+0000","last_unstale":"2024-02-26T06:26:13.874338+0000","last_undegraded":"2024-02-26T06:26:13.874338+0000","last_fullsized":"2024-02-26T06:26:13.874338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:20.887692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.117029+0000","last_change":"2024-02-26T06:26:13.864818+0000","last_active":"2024-02-26T06:26:42.117029+0000","last_peered":"2024-02-26T06:26:42.117029+0000","last_clean":"2024-02-26T06:26:42.117029+0000","last_became_active":"2024-02-26T06:26:13.863821+0000","last_became_peered":"2024-02-26T06:26:13.863821+0000","last_unstale":"2024-02-26T06:26:42.117029+0000","last_undegraded":"2024-02-26T06:26:42.117029+0000","last_fullsized":"2024-02-26T06:26:42.117029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:38.266114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":172,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005658+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:27:49.005658+0000","last_peered":"2024-02-26T06:27:49.005658+0000","last_clean":"2024-02-26T06:27:49.005658+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:27:49.005658+0000","last_undegraded":"2024-02-26T06:27:49.005658+0000","last_fullsized":"2024-02-26T06:27:49.005658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098872+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:26:42.098872+0000","last_peered":"2024-02-26T06:26:42.098872+0000","last_clean":"2024-02-26T06:26:42.098872+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:26:42.098872+0000","last_undegraded":"2024-02-26T06:26:42.098872+0000","last_fullsized":"2024-02-26T06:26:42.098872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851763+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:26:13.851763+0000","last_peered":"2024-02-26T06:26:13.851763+0000","last_clean":"2024-02-26T06:26:13.851763+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:26:13.851763+0000","last_undegraded":"2024-02-26T06:26:13.851763+0000","last_fullsized":"2024-02-26T06:26:13.851763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131161+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:26:42.131161+0000","last_peered":"2024-02-26T06:26:42.131161+0000","last_clean":"2024-02-26T06:26:42.131161+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:26:42.131161+0000","last_undegraded":"2024-02-26T06:26:42.131161+0000","last_fullsized":"2024-02-26T06:26:42.131161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.858385+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:26:13.858385+0000","last_peered":"2024-02-26T06:26:13.858385+0000","last_clean":"2024-02-26T06:26:13.858385+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:26:13.858385+0000","last_undegraded":"2024-02-26T06:26:13.858385+0000","last_fullsized":"2024-02-26T06:26:13.858385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128140+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:26:42.128140+0000","last_peered":"2024-02-26T06:26:42.128140+0000","last_clean":"2024-02-26T06:26:42.128140+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:26:42.128140+0000","last_undegraded":"2024-02-26T06:26:42.128140+0000","last_fullsized":"2024-02-26T06:26:42.128140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.850787+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:26:13.850787+0000","last_peered":"2024-02-26T06:26:13.850787+0000","last_clean":"2024-02-26T06:26:13.850787+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:26:13.850787+0000","last_undegraded":"2024-02-26T06:26:13.850787+0000","last_fullsized":"2024-02-26T06:26:13.850787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131708+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:26:42.131708+0000","last_peered":"2024-02-26T06:26:42.131708+0000","last_clean":"2024-02-26T06:26:42.131708+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:26:42.131708+0000","last_undegraded":"2024-02-26T06:26:42.131708+0000","last_fullsized":"2024-02-26T06:26:42.131708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":315,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005095+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:27:49.005095+0000","last_peered":"2024-02-26T06:27:49.005095+0000","last_clean":"2024-02-26T06:27:49.005095+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:27:49.005095+0000","last_undegraded":"2024-02-26T06:27:49.005095+0000","last_fullsized":"2024-02-26T06:27:49.005095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":290,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.115755+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:26:42.115755+0000","last_peered":"2024-02-26T06:26:42.115755+0000","last_clean":"2024-02-26T06:26:42.115755+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:26:42.115755+0000","last_undegraded":"2024-02-26T06:26:42.115755+0000","last_fullsized":"2024-02-26T06:26:42.115755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848306+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:26:13.848306+0000","last_peered":"2024-02-26T06:26:13.848306+0000","last_clean":"2024-02-26T06:26:13.848306+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:26:13.848306+0000","last_undegraded":"2024-02-26T06:26:13.848306+0000","last_fullsized":"2024-02-26T06:26:13.848306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128679+0000","last_change":"2024-02-26T06:26:13.858594+0000","last_active":"2024-02-26T06:26:42.128679+0000","last_peered":"2024-02-26T06:26:42.128679+0000","last_clean":"2024-02-26T06:26:42.128679+0000","last_became_active":"2024-02-26T06:26:13.852999+0000","last_became_peered":"2024-02-26T06:26:13.852999+0000","last_unstale":"2024-02-26T06:26:42.128679+0000","last_undegraded":"2024-02-26T06:26:42.128679+0000","last_fullsized":"2024-02-26T06:26:42.128679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:17:51.940824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851822+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:26:13.851822+0000","last_peered":"2024-02-26T06:26:13.851822+0000","last_clean":"2024-02-26T06:26:13.851822+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:26:13.851822+0000","last_undegraded":"2024-02-26T06:26:13.851822+0000","last_fullsized":"2024-02-26T06:26:13.851822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125874+0000","last_change":"2024-02-26T06:26:13.863050+0000","last_active":"2024-02-26T06:26:42.125874+0000","last_peered":"2024-02-26T06:26:42.125874+0000","last_clean":"2024-02-26T06:26:42.125874+0000","last_became_active":"2024-02-26T06:26:13.862635+0000","last_became_peered":"2024-02-26T06:26:13.862635+0000","last_unstale":"2024-02-26T06:26:42.125874+0000","last_undegraded":"2024-02-26T06:26:42.125874+0000","last_fullsized":"2024-02-26T06:26:42.125874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:38:45.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":360,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005625+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:27:49.005625+0000","last_peered":"2024-02-26T06:27:49.005625+0000","last_clean":"2024-02-26T06:27:49.005625+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:27:49.005625+0000","last_undegraded":"2024-02-26T06:27:49.005625+0000","last_fullsized":"2024-02-26T06:27:49.005625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66346,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":301,"num_read_kb":216,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100074+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:26:42.100074+0000","last_peered":"2024-02-26T06:26:42.100074+0000","last_clean":"2024-02-26T06:26:42.100074+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:26:42.100074+0000","last_undegraded":"2024-02-26T06:26:42.100074+0000","last_fullsized":"2024-02-26T06:26:42.100074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848123+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:26:13.848123+0000","last_peered":"2024-02-26T06:26:13.848123+0000","last_clean":"2024-02-26T06:26:13.848123+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:26:13.848123+0000","last_undegraded":"2024-02-26T06:26:13.848123+0000","last_fullsized":"2024-02-26T06:26:13.848123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131830+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:26:42.131830+0000","last_peered":"2024-02-26T06:26:42.131830+0000","last_clean":"2024-02-26T06:26:42.131830+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:26:42.131830+0000","last_undegraded":"2024-02-26T06:26:42.131830+0000","last_fullsized":"2024-02-26T06:26:42.131830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851163+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:26:13.851163+0000","last_peered":"2024-02-26T06:26:13.851163+0000","last_clean":"2024-02-26T06:26:13.851163+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:26:13.851163+0000","last_undegraded":"2024-02-26T06:26:13.851163+0000","last_fullsized":"2024-02-26T06:26:13.851163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112311+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:26:42.112311+0000","last_peered":"2024-02-26T06:26:42.112311+0000","last_clean":"2024-02-26T06:26:42.112311+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:26:42.112311+0000","last_undegraded":"2024-02-26T06:26:42.112311+0000","last_fullsized":"2024-02-26T06:26:42.112311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'8","reported_seq":177,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:48.128168+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:27:48.128168+0000","last_peered":"2024-02-26T06:27:48.128168+0000","last_clean":"2024-02-26T06:27:48.128168+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:27:48.128168+0000","last_undegraded":"2024-02-26T06:27:48.128168+0000","last_fullsized":"2024-02-26T06:27:48.128168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113095+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:26:42.113095+0000","last_peered":"2024-02-26T06:26:42.113095+0000","last_clean":"2024-02-26T06:26:42.113095+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:26:42.113095+0000","last_undegraded":"2024-02-26T06:26:42.113095+0000","last_fullsized":"2024-02-26T06:26:42.113095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844759+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:26:13.844759+0000","last_peered":"2024-02-26T06:26:13.844759+0000","last_clean":"2024-02-26T06:26:13.844759+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:26:13.844759+0000","last_undegraded":"2024-02-26T06:26:13.844759+0000","last_fullsized":"2024-02-26T06:26:13.844759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113707+0000","last_change":"2024-02-26T06:26:13.862455+0000","last_active":"2024-02-26T06:26:42.113707+0000","last_peered":"2024-02-26T06:26:42.113707+0000","last_clean":"2024-02-26T06:26:42.113707+0000","last_became_active":"2024-02-26T06:26:13.861385+0000","last_became_peered":"2024-02-26T06:26:13.861385+0000","last_unstale":"2024-02-26T06:26:42.113707+0000","last_undegraded":"2024-02-26T06:26:42.113707+0000","last_fullsized":"2024-02-26T06:26:42.113707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:42:41.345104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869092+0000","last_change":"2024-02-26T06:26:13.868540+0000","last_active":"2024-02-26T06:26:13.869092+0000","last_peered":"2024-02-26T06:26:13.869092+0000","last_clean":"2024-02-26T06:26:13.869092+0000","last_became_active":"2024-02-26T06:26:13.852149+0000","last_became_peered":"2024-02-26T06:26:13.852149+0000","last_unstale":"2024-02-26T06:26:13.869092+0000","last_undegraded":"2024-02-26T06:26:13.869092+0000","last_fullsized":"2024-02-26T06:26:13.869092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:14:30.745782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101726+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:26:42.101726+0000","last_peered":"2024-02-26T06:26:42.101726+0000","last_clean":"2024-02-26T06:26:42.101726+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:26:42.101726+0000","last_undegraded":"2024-02-26T06:26:42.101726+0000","last_fullsized":"2024-02-26T06:26:42.101726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.860103+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:26:13.860103+0000","last_peered":"2024-02-26T06:26:13.860103+0000","last_clean":"2024-02-26T06:26:13.860103+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:26:13.860103+0000","last_undegraded":"2024-02-26T06:26:13.860103+0000","last_fullsized":"2024-02-26T06:26:13.860103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105524+0000","last_change":"2024-02-26T06:26:13.872244+0000","last_active":"2024-02-26T06:26:42.105524+0000","last_peered":"2024-02-26T06:26:42.105524+0000","last_clean":"2024-02-26T06:26:42.105524+0000","last_became_active":"2024-02-26T06:26:13.866094+0000","last_became_peered":"2024-02-26T06:26:13.866094+0000","last_unstale":"2024-02-26T06:26:42.105524+0000","last_undegraded":"2024-02-26T06:26:42.105524+0000","last_fullsized":"2024-02-26T06:26:42.105524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:56:52.301479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":167,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:44.106988+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:27:44.106988+0000","last_peered":"2024-02-26T06:27:44.106988+0000","last_clean":"2024-02-26T06:27:44.106988+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:27:44.106988+0000","last_undegraded":"2024-02-26T06:27:44.106988+0000","last_fullsized":"2024-02-26T06:27:44.106988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125601+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:26:42.125601+0000","last_peered":"2024-02-26T06:26:42.125601+0000","last_clean":"2024-02-26T06:26:42.125601+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:26:42.125601+0000","last_undegraded":"2024-02-26T06:26:42.125601+0000","last_fullsized":"2024-02-26T06:26:42.125601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.094140+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:26:42.094140+0000","last_peered":"2024-02-26T06:26:42.094140+0000","last_clean":"2024-02-26T06:26:42.094140+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:26:42.094140+0000","last_undegraded":"2024-02-26T06:26:42.094140+0000","last_fullsized":"2024-02-26T06:26:42.094140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":168,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:44.116835+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:27:44.116835+0000","last_peered":"2024-02-26T06:27:44.116835+0000","last_clean":"2024-02-26T06:27:44.116835+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:27:44.116835+0000","last_undegraded":"2024-02-26T06:27:44.116835+0000","last_fullsized":"2024-02-26T06:27:44.116835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130696+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:26:42.130696+0000","last_peered":"2024-02-26T06:26:42.130696+0000","last_clean":"2024-02-26T06:26:42.130696+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:26:42.130696+0000","last_undegraded":"2024-02-26T06:26:42.130696+0000","last_fullsized":"2024-02-26T06:26:42.130696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849961+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:26:13.849961+0000","last_peered":"2024-02-26T06:26:13.849961+0000","last_clean":"2024-02-26T06:26:13.849961+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:26:13.849961+0000","last_undegraded":"2024-02-26T06:26:13.849961+0000","last_fullsized":"2024-02-26T06:26:13.849961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846775+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:26:13.846775+0000","last_peered":"2024-02-26T06:26:13.846775+0000","last_clean":"2024-02-26T06:26:13.846775+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:26:13.846775+0000","last_undegraded":"2024-02-26T06:26:13.846775+0000","last_fullsized":"2024-02-26T06:26:13.846775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121139+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:26:42.121139+0000","last_peered":"2024-02-26T06:26:42.121139+0000","last_clean":"2024-02-26T06:26:42.121139+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:26:42.121139+0000","last_undegraded":"2024-02-26T06:26:42.121139+0000","last_fullsized":"2024-02-26T06:26:42.121139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846885+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:26:13.846885+0000","last_peered":"2024-02-26T06:26:13.846885+0000","last_clean":"2024-02-26T06:26:13.846885+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:26:13.846885+0000","last_undegraded":"2024-02-26T06:26:13.846885+0000","last_fullsized":"2024-02-26T06:26:13.846885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.845086+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:26:13.845086+0000","last_peered":"2024-02-26T06:26:13.845086+0000","last_clean":"2024-02-26T06:26:13.845086+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:26:13.845086+0000","last_undegraded":"2024-02-26T06:26:13.845086+0000","last_fullsized":"2024-02-26T06:26:13.845086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123071+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:26:42.123071+0000","last_peered":"2024-02-26T06:26:42.123071+0000","last_clean":"2024-02-26T06:26:42.123071+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:26:42.123071+0000","last_undegraded":"2024-02-26T06:26:42.123071+0000","last_fullsized":"2024-02-26T06:26:42.123071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:45.258314+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:27:45.258314+0000","last_peered":"2024-02-26T06:27:45.258314+0000","last_clean":"2024-02-26T06:27:45.258314+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:27:45.258314+0000","last_undegraded":"2024-02-26T06:27:45.258314+0000","last_fullsized":"2024-02-26T06:27:45.258314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126193+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:26:42.126193+0000","last_peered":"2024-02-26T06:26:42.126193+0000","last_clean":"2024-02-26T06:26:42.126193+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:26:42.126193+0000","last_undegraded":"2024-02-26T06:26:42.126193+0000","last_fullsized":"2024-02-26T06:26:42.126193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849969+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:26:13.849969+0000","last_peered":"2024-02-26T06:26:13.849969+0000","last_clean":"2024-02-26T06:26:13.849969+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:26:13.849969+0000","last_undegraded":"2024-02-26T06:26:13.849969+0000","last_fullsized":"2024-02-26T06:26:13.849969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128571+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:26:42.128571+0000","last_peered":"2024-02-26T06:26:42.128571+0000","last_clean":"2024-02-26T06:26:42.128571+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:26:42.128571+0000","last_undegraded":"2024-02-26T06:26:42.128571+0000","last_fullsized":"2024-02-26T06:26:42.128571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.847127+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:26:13.847127+0000","last_peered":"2024-02-26T06:26:13.847127+0000","last_clean":"2024-02-26T06:26:13.847127+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:26:13.847127+0000","last_undegraded":"2024-02-26T06:26:13.847127+0000","last_fullsized":"2024-02-26T06:26:13.847127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098805+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:26:42.098805+0000","last_peered":"2024-02-26T06:26:42.098805+0000","last_clean":"2024-02-26T06:26:42.098805+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:26:42.098805+0000","last_undegraded":"2024-02-26T06:26:42.098805+0000","last_fullsized":"2024-02-26T06:26:42.098805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.866330+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:26:13.866330+0000","last_peered":"2024-02-26T06:26:13.866330+0000","last_clean":"2024-02-26T06:26:13.866330+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:26:13.866330+0000","last_undegraded":"2024-02-26T06:26:13.866330+0000","last_fullsized":"2024-02-26T06:26:13.866330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125635+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:26:42.125635+0000","last_peered":"2024-02-26T06:26:42.125635+0000","last_clean":"2024-02-26T06:26:42.125635+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:26:42.125635+0000","last_undegraded":"2024-02-26T06:26:42.125635+0000","last_fullsized":"2024-02-26T06:26:42.125635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848328+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:26:13.848328+0000","last_peered":"2024-02-26T06:26:13.848328+0000","last_clean":"2024-02-26T06:26:13.848328+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:26:13.848328+0000","last_undegraded":"2024-02-26T06:26:13.848328+0000","last_fullsized":"2024-02-26T06:26:13.848328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129822+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:26:42.129822+0000","last_peered":"2024-02-26T06:26:42.129822+0000","last_clean":"2024-02-26T06:26:42.129822+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:26:42.129822+0000","last_undegraded":"2024-02-26T06:26:42.129822+0000","last_fullsized":"2024-02-26T06:26:42.129822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844858+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:26:13.844858+0000","last_peered":"2024-02-26T06:26:13.844858+0000","last_clean":"2024-02-26T06:26:13.844858+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:26:13.844858+0000","last_undegraded":"2024-02-26T06:26:13.844858+0000","last_fullsized":"2024-02-26T06:26:13.844858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112189+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:26:42.112189+0000","last_peered":"2024-02-26T06:26:42.112189+0000","last_clean":"2024-02-26T06:26:42.112189+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:26:42.112189+0000","last_undegraded":"2024-02-26T06:26:42.112189+0000","last_fullsized":"2024-02-26T06:26:42.112189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":170,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:45.000131+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:27:45.000131+0000","last_peered":"2024-02-26T06:27:45.000131+0000","last_clean":"2024-02-26T06:27:45.000131+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:27:45.000131+0000","last_undegraded":"2024-02-26T06:27:45.000131+0000","last_fullsized":"2024-02-26T06:27:45.000131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1480,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107689+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:26:42.107689+0000","last_peered":"2024-02-26T06:26:42.107689+0000","last_clean":"2024-02-26T06:26:42.107689+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:26:42.107689+0000","last_undegraded":"2024-02-26T06:26:42.107689+0000","last_fullsized":"2024-02-26T06:26:42.107689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851673+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:26:13.851673+0000","last_peered":"2024-02-26T06:26:13.851673+0000","last_clean":"2024-02-26T06:26:13.851673+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:26:13.851673+0000","last_undegraded":"2024-02-26T06:26:13.851673+0000","last_fullsized":"2024-02-26T06:26:13.851673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.111993+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:26:42.111993+0000","last_peered":"2024-02-26T06:26:42.111993+0000","last_clean":"2024-02-26T06:26:42.111993+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:26:42.111993+0000","last_undegraded":"2024-02-26T06:26:42.111993+0000","last_fullsized":"2024-02-26T06:26:42.111993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.879538+0000","last_change":"2024-02-26T06:26:13.878533+0000","last_active":"2024-02-26T06:26:13.879538+0000","last_peered":"2024-02-26T06:26:13.879538+0000","last_clean":"2024-02-26T06:26:13.879538+0000","last_became_active":"2024-02-26T06:26:13.866478+0000","last_became_peered":"2024-02-26T06:26:13.866478+0000","last_unstale":"2024-02-26T06:26:13.879538+0000","last_undegraded":"2024-02-26T06:26:13.879538+0000","last_fullsized":"2024-02-26T06:26:13.879538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:00:23.896436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099320+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:26:42.099320+0000","last_peered":"2024-02-26T06:26:42.099320+0000","last_clean":"2024-02-26T06:26:42.099320+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:26:42.099320+0000","last_undegraded":"2024-02-26T06:26:42.099320+0000","last_fullsized":"2024-02-26T06:26:42.099320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100022+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:26:42.100022+0000","last_peered":"2024-02-26T06:26:42.100022+0000","last_clean":"2024-02-26T06:26:42.100022+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:26:42.100022+0000","last_undegraded":"2024-02-26T06:26:42.100022+0000","last_fullsized":"2024-02-26T06:26:42.100022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.108252+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:26:42.108252+0000","last_peered":"2024-02-26T06:26:42.108252+0000","last_clean":"2024-02-26T06:26:42.108252+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:26:42.108252+0000","last_undegraded":"2024-02-26T06:26:42.108252+0000","last_fullsized":"2024-02-26T06:26:42.108252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124761+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:26:42.124761+0000","last_peered":"2024-02-26T06:26:42.124761+0000","last_clean":"2024-02-26T06:26:42.124761+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:26:42.124761+0000","last_undegraded":"2024-02-26T06:26:42.124761+0000","last_fullsized":"2024-02-26T06:26:42.124761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127390+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:26:42.127390+0000","last_peered":"2024-02-26T06:26:42.127390+0000","last_clean":"2024-02-26T06:26:42.127390+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:26:42.127390+0000","last_undegraded":"2024-02-26T06:26:42.127390+0000","last_fullsized":"2024-02-26T06:26:42.127390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126867+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:26:42.126867+0000","last_peered":"2024-02-26T06:26:42.126867+0000","last_clean":"2024-02-26T06:26:42.126867+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:26:42.126867+0000","last_undegraded":"2024-02-26T06:26:42.126867+0000","last_fullsized":"2024-02-26T06:26:42.126867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.096844+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:26:42.096844+0000","last_peered":"2024-02-26T06:26:42.096844+0000","last_clean":"2024-02-26T06:26:42.096844+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:26:42.096844+0000","last_undegraded":"2024-02-26T06:26:42.096844+0000","last_fullsized":"2024-02-26T06:26:42.096844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106293+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:26:42.106293+0000","last_peered":"2024-02-26T06:26:42.106293+0000","last_clean":"2024-02-26T06:26:42.106293+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:26:42.106293+0000","last_undegraded":"2024-02-26T06:26:42.106293+0000","last_fullsized":"2024-02-26T06:26:42.106293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112460+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:26:42.112460+0000","last_peered":"2024-02-26T06:26:42.112460+0000","last_clean":"2024-02-26T06:26:42.112460+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:26:42.112460+0000","last_undegraded":"2024-02-26T06:26:42.112460+0000","last_fullsized":"2024-02-26T06:26:42.112460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130712+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:26:42.130712+0000","last_peered":"2024-02-26T06:26:42.130712+0000","last_clean":"2024-02-26T06:26:42.130712+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:26:42.130712+0000","last_undegraded":"2024-02-26T06:26:42.130712+0000","last_fullsized":"2024-02-26T06:26:42.130712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125683+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:26:42.125683+0000","last_peered":"2024-02-26T06:26:42.125683+0000","last_clean":"2024-02-26T06:26:42.125683+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:26:42.125683+0000","last_undegraded":"2024-02-26T06:26:42.125683+0000","last_fullsized":"2024-02-26T06:26:42.125683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131704+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:26:42.131704+0000","last_peered":"2024-02-26T06:26:42.131704+0000","last_clean":"2024-02-26T06:26:42.131704+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:26:42.131704+0000","last_undegraded":"2024-02-26T06:26:42.131704+0000","last_fullsized":"2024-02-26T06:26:42.131704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129590+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:26:42.129590+0000","last_peered":"2024-02-26T06:26:42.129590+0000","last_clean":"2024-02-26T06:26:42.129590+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:26:42.129590+0000","last_undegraded":"2024-02-26T06:26:42.129590+0000","last_fullsized":"2024-02-26T06:26:42.129590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100819+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:26:42.100819+0000","last_peered":"2024-02-26T06:26:42.100819+0000","last_clean":"2024-02-26T06:26:42.100819+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:26:42.100819+0000","last_undegraded":"2024-02-26T06:26:42.100819+0000","last_fullsized":"2024-02-26T06:26:42.100819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74001,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1480,"num_read_kb":823,"num_write":258,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":216239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94699999999999995,"5min":1.113,"15min":1.113},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.748}]},{"osd":1,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59699999999999998,"15min":0.56699999999999995},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.36499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66000000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.51900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.751,"15min":0.751},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.71299999999999997,"5min":0.874,"15min":0.874},"last":0.51800000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.84999999999999998},"min":{"1min":0.56299999999999994,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.64600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":1.385,"15min":1.385},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.57999999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.623},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69699999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.90100000000000002,"15min":0.90000000000000002},"min":{"1min":0.67400000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.179,"5min":1.238,"15min":1.238},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.95099999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.151,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.58299999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":1.044,"15min":1.044},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.73099999999999998},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.93500000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71799999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1115232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":0.93700000000000006},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.4740000000000002,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45500000000000002,"15min":0.46300000000000002},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.373},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.627,"5min":1.093,"15min":1.093},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.752,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51600000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.167,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0920000000000001,"5min":1.458,"15min":1.458},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.52500000000000002,"15min":0.54300000000000004},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.56299999999999994,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.69499999999999995,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76900000000000002,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81100000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.73999999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82499999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60599999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.004,"15min":1.034},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1279999999999999,"5min":1.256,"15min":1.256},"last":0.98699999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77100000000000002,"15min":0.78000000000000003},"min":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.629}]}]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.367,"15min":1.419},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.8979999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.3700000000000001,"15min":1.429},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7889999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.238}]},{"osd":1,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.1020000000000001,"15min":1.1759999999999999},"min":{"1min":0.70599999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.266,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.675},{"interface":"front","average":{"1min":1.117,"5min":1.2090000000000001,"15min":1.292},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.393,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1539999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4670000000000001,"15min":1.5429999999999999},"min":{"1min":0.69799999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8759999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3320000000000001,"15min":1.3919999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0209999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.468}]},{"osd":3,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.613,"15min":1.7350000000000001},"min":{"1min":0.749,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1400000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.736},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.7410000000000001,"15min":1.839},"min":{"1min":0.89800000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1869999999999998,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":2.238}]},{"osd":4,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7470000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1859999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":2.2669999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.766,"15min":1.9159999999999999},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":2.665}]},{"osd":5,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.7709999999999999,"5min":2.012,"15min":2.0699999999999998},"min":{"1min":1.347,"5min":0.879,"15min":0.879},"max":{"1min":2.298,"5min":14.581,"15min":14.581},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.708,"5min":2.3500000000000001,"15min":2.7480000000000002},"min":{"1min":1.268,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3650000000000002,"5min":18.216000000000001,"15min":18.216000000000001},"last":2.3170000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.117,"15min":2.2599999999999998},"min":{"1min":1.2749999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.4199999999999999,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":3.4209999999999998},{"interface":"front","average":{"1min":1.8089999999999999,"5min":2.0339999999999998,"15min":2.194},"min":{"1min":1.167,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.5899999999999999,"5min":3.044,"15min":3.044},"last":3.5619999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.9339999999999999,"15min":2.0619999999999998},"min":{"1min":1.071,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.2269999999999999,"5min":3.036,"15min":3.036},"last":3.0870000000000002},{"interface":"front","average":{"1min":1.875,"5min":2.0510000000000002,"15min":2.1669999999999998},"min":{"1min":1.1499999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.4870000000000001,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":2.1230000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":2.0529999999999999,"5min":2.1589999999999998,"15min":2.2949999999999999},"min":{"1min":1.3859999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"max":{"1min":2.6160000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.1970000000000001},{"interface":"front","average":{"1min":1.889,"5min":2.0249999999999999,"15min":2.1579999999999999},"min":{"1min":1.359,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.492,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":2.355}]},{"osd":10,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":1.895,"5min":1.9390000000000001,"15min":1.9590000000000001},"min":{"1min":1.0389999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.4609999999999999,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":2.9630000000000001},{"interface":"front","average":{"1min":2.0939999999999999,"5min":2.1669999999999998,"15min":2.1949999999999998},"min":{"1min":1.089,"5min":1.089,"15min":1.089},"max":{"1min":2.6629999999999998,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":2.1619999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.44500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.505,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3069999999999999,"5min":1.478,"15min":1.478},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.47699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.752}]},{"osd":2,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.442},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45800000000000002,"15min":0.442},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.77400000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.55200000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1379999999999999,"5min":1.216,"15min":1.216},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1919999999999999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90000000000000002,"15min":0.92200000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.373,"15min":1.373},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.67700000000000005},"min":{"1min":0.56499999999999995,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64300000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91500000000000004,"15min":0.91800000000000004},"min":{"1min":0.78100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.284,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.97499999999999998,"15min":1.0069999999999999},"min":{"1min":0.72899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.079}]},{"osd":10,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.73999999999999999},"min":{"1min":0.60499999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0329999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.55400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.59599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.48799999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70699999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1049901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.40600000000000003},"min":{"1min":0.32000000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.52200000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.46300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.879,"15min":0.879},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.1240000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.497,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.495},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.76700000000000002,"5min":0.877,"15min":0.877},"last":0.79800000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.90700000000000003,"15min":0.86899999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94999999999999996,"5min":6.9900000000000002,"15min":6.9900000000000002},"last":0.88500000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79700000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78300000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":0.99199999999999999},"min":{"1min":0.73099999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.038},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0649999999999999,"15min":1.038},"min":{"1min":0.66800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1839999999999999,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.91900000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73899999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.66100000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.90000000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82899999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.204,"15min":1.1970000000000001},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.923,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3080000000000001,"15min":1.3069999999999999},"min":{"1min":0.55900000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1749999999999998,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.1180000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0700000000000001,"15min":1.036},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6870000000000001,"5min":1.867,"15min":1.867},"last":1.073},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.1519999999999999,"15min":1.117},"min":{"1min":0.69099999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.4750000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.347}]},{"osd":2,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.23,"15min":1.1559999999999999},"min":{"1min":0.94199999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.27},{"interface":"front","average":{"1min":1.331,"5min":1.3009999999999999,"15min":1.2370000000000001},"min":{"1min":0.65800000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.2530000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.488}]},{"osd":3,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.579,"15min":1.5369999999999999},"min":{"1min":0.82499999999999996,"5min":0.501,"15min":0.501},"max":{"1min":2.5830000000000002,"5min":3.113,"15min":3.113},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.625,"15min":1.647},"min":{"1min":0.71199999999999997,"5min":0.496,"15min":0.496},"max":{"1min":3.294,"5min":3.294,"15min":3.294},"last":2.0259999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.381,"15min":1.359},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5740000000000001,"15min":1.514},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.492,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.6559999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5640000000000001,"15min":1.613},"min":{"1min":0.95699999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7210000000000001,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":2.141},{"interface":"front","average":{"1min":1.657,"5min":1.619,"15min":1.6040000000000001},"min":{"1min":0.91600000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.786,"5min":2.798,"15min":2.798},"last":2.0990000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:27:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.7410000000000001,"15min":1.571},"min":{"1min":1.0049999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.6669999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":1.784},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.7030000000000001,"15min":1.625},"min":{"1min":0.72699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1760000000000002,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":2.2050000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.827,"15min":1.7410000000000001},"min":{"1min":1.1080000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":3.3620000000000001,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9410000000000001},{"interface":"front","average":{"1min":1.867,"5min":1.7869999999999999,"15min":1.7509999999999999},"min":{"1min":1.0569999999999999,"5min":0.747,"15min":0.747},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":2.2839999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:27:28 2024","interfaces":[{"interface":"back","average":{"1min":2.004,"5min":2.04,"15min":2.234},"min":{"1min":0.94799999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":2.242},{"interface":"front","average":{"1min":2.133,"5min":2.157,"15min":2.3380000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":3.1000000000000001,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":2.0609999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9339999999999999,"5min":2.024,"15min":2.113},"min":{"1min":1.1679999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":3.2330000000000001,"5min":3.2330000000000001,"15min":3.2330000000000001},"last":2.3599999999999999},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.9670000000000001,"15min":2.008},"min":{"1min":1.042,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.1829999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":523449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.627},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.374},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0740000000000001,"5min":1.252,"15min":1.252},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Feb 26 06:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95799999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:26:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81599999999999995,"15min":0.80100000000000005},"min":{"1min":0.55200000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1639999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.34599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.43099999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55900000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.86699999999999999},"min":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.077,"5min":1.196,"15min":1.196},"last":0.879},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.90900000000000003},"min":{"1min":0.61399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.21,"15min":1.21},"last":0.96499999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.498,"15min":0.495},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67400000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46600000000000003}]},{"osd":8,"last update":"Mon Feb 26 06:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.622}]},{"osd":9,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.84499999999999997},"min":{"1min":0.495,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.254,"5min":1.474,"15min":1.474},"last":0.748},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.92900000000000005},"min":{"1min":0.69599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.92700000000000005}]},{"osd":10,"last update":"Mon Feb 26 06:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.496}]},{"osd":11,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66000000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":458621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.96599999999999997,"15min":0.81599999999999995},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.45900000000000002},"max":{"1min":1.379,"5min":1.569,"15min":1.569},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0069999999999999,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.40899999999999997},"max":{"1min":1.3320000000000001,"5min":3.516,"15min":3.516},"last":0.94099999999999995}]},{"osd":2,"last update":"Mon Feb 26 06:27:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0649999999999999,"15min":0.92200000000000004},"min":{"1min":0.64700000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2190000000000001,"15min":1.0109999999999999},"min":{"1min":0.89900000000000002,"5min":0.495,"15min":0.48599999999999999},"max":{"1min":1.9419999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3600000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.6519999999999999,"15min":1.6679999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.3010000000000002,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.113},{"interface":"front","average":{"1min":1.782,"5min":1.794,"15min":1.796},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.3100000000000001,"5min":2.718,"15min":2.718},"last":1.486}]},{"osd":4,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.3460000000000001},"min":{"1min":0.78800000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.1160000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.139,"15min":1.133},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.621,"5min":2.069,"15min":2.069},"last":0.84799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:27:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3660000000000001,"15min":1.3220000000000001},"min":{"1min":0.77700000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.9059999999999999,"5min":2.653,"15min":2.653},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.7210000000000001,"15min":1.5600000000000001},"min":{"1min":0.97299999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.5710000000000002,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.431}]},{"osd":6,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.532,"15min":1.4530000000000001},"min":{"1min":0.745,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.46,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6890000000000001,"15min":1.575},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.383}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.7110000000000001,"15min":1.671},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.2749999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.294},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.875,"15min":1.8089999999999999},"min":{"1min":0.91200000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.552,"5min":2.911,"15min":2.911},"last":1.5229999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:27:30 2024","interfaces":[{"interface":"back","average":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.173},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.254,"5min":3.254,"15min":3.254},"last":1.4610000000000001},{"interface":"front","average":{"1min":2.0950000000000002,"5min":2.1629999999999998,"15min":2.2690000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.7999999999999998,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.5620000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0259999999999998,"5min":2.0960000000000001,"15min":2.137},"min":{"1min":1.1200000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.8620000000000001,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.673,"15min":1.7270000000000001},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":2.2290000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.196}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.8600000000000001,"15min":1.8700000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4180000000000001,"5min":2.524,"15min":2.524},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6739999999999999,"15min":1.6819999999999999},"min":{"1min":0.89500000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.4500000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.403}]}]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":518606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72699999999999998,"15min":0.67100000000000004},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.163,"15min":1.163},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.68000000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1879999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Feb 26 06:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80700000000000005,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75600000000000001,"15min":0.69899999999999995},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.94399999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.505,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.39300000000000002},"min":{"1min":0.30299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85199999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.42899999999999999},"min":{"1min":0.35399999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42199999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88600000000000001,"15min":0.88500000000000001},"min":{"1min":0.74399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.071},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.90400000000000003,"15min":0.86499999999999999},"min":{"1min":0.64600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.319,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.51300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.89000000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63800000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.86399999999999999,"15min":0.86199999999999999},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.04,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.753},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.93200000000000005,"15min":0.95799999999999996},"min":{"1min":0.67400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1850000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.0569999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.628},"min":{"1min":0.55600000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.621},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.089,"15min":1.089},"last":0.55900000000000005}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0369999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73499999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":456289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.1799999999999999,"15min":0.97899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.45,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.45},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.369,"15min":1.0549999999999999},"min":{"1min":0.71799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8440000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.6080000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.1970000000000001,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.5199999999999996,"5min":7.5199999999999996,"15min":7.5199999999999996},"last":1.224},{"interface":"front","average":{"1min":1.143,"5min":1.2010000000000001,"15min":0.997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8300000000000001,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.3129999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.252,"15min":1.077},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.748,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.262},{"interface":"front","average":{"1min":1.74,"5min":1.397,"15min":1.171},"min":{"1min":0.50800000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3760000000000003,"5min":7.3760000000000003,"15min":7.3760000000000003},"last":1.484}]},{"osd":4,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3380000000000001,"15min":1.1240000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.9450000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.028},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.4550000000000001,"15min":1.258},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0979999999999999,"5min":3.323,"15min":3.323},"last":1.411}]},{"osd":5,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.5960000000000001,"15min":1.391},"min":{"1min":0.84499999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.4039999999999999,"5min":7.4039999999999999,"15min":7.4039999999999999},"last":1.365},{"interface":"front","average":{"1min":1.51,"5min":1.5600000000000001,"15min":1.51},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4199999999999999,"5min":3.1349999999999998,"15min":5.7770000000000001},"last":1.54}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":1.829,"15min":1.7390000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2949999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.766},{"interface":"front","average":{"1min":1.556,"5min":1.671,"15min":1.4990000000000001},"min":{"1min":0.64600000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.2170000000000001,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.7490000000000001,"15min":1.7010000000000001},"min":{"1min":0.80000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2829999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.768,"5min":1.8999999999999999,"15min":1.8480000000000001},"min":{"1min":0.75,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3149999999999999,"5min":6.758,"15min":6.758},"last":1.5940000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.9870000000000001,"15min":2.145},"min":{"1min":0.72799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.6200000000000001,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.51},{"interface":"front","average":{"1min":1.847,"5min":2.0819999999999999,"15min":2.2349999999999999},"min":{"1min":0.95299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.5449999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":1.837}]},{"osd":10,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":2.0750000000000002,"15min":2.1789999999999998},"min":{"1min":0.77400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.669,"5min":2.0049999999999999,"15min":2.153},"min":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.8140000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.6870000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.929,"5min":1.8320000000000001,"15min":1.8160000000000001},"min":{"1min":1.1919999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.6549999999999998,"5min":3.085,"15min":3.085},"last":1.74},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7090000000000001,"15min":1.6910000000000001},"min":{"1min":0.79600000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.5059999999999998,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.7889999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.41199999999999998,"15min":0.38200000000000001},"min":{"1min":0.32500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.752,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.745},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.54300000000000004},"min":{"1min":0.46600000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":1.04,"15min":1.04},"last":0.5}]},{"osd":3,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78500000000000003,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.022,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":0.71499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1439999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.56899999999999995},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82699999999999996,"15min":0.79100000000000004},"min":{"1min":0.48999999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.02,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.82999999999999996,"15min":0.84399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1659999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.48299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85199999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92200000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66600000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86299999999999999,"15min":0.89500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.173,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94299999999999995,"15min":0.99299999999999999},"min":{"1min":0.69599999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.0580000000000001,"5min":1.665,"15min":1.665},"last":0.59899999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68500000000000005,"15min":0.67400000000000004},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89500000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.47099999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78200000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88400000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.81399999999999995,"15min":0.80900000000000005},"min":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.73299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1049047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73899999999999999},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1339999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.754,"5min":0.76600000000000001,"15min":0.95699999999999996},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.03,"5min":1.99,"15min":7.2249999999999996},"last":0.73599999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.21,"15min":1.21},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.5,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84499999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.54200000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.44},{"interface":"front","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.629,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.74399999999999999},"min":{"1min":0.54000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1830000000000001,"5min":1.298,"15min":1.298},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.82599999999999996,"15min":0.76600000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81499999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80500000000000005,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.46200000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81899999999999995,"15min":0.85099999999999998},"min":{"1min":0.60499999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.075,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.65700000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.109,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.196,"15min":1.196},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.55500000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":1.177,"15min":1.177},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.441,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.40600000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.502}]},{"osd":9,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93200000000000005,"15min":0.89100000000000001},"min":{"1min":0.73299999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.1910000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0129999999999999,"15min":0.996},"min":{"1min":0.79800000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.266,"5min":1.667,"15min":1.667},"last":0.76500000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.53700000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.413,"15min":1.413},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:27:51.510 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:27:51.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 -- 172.21.15.6:0/2179704938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0101210 msgr2=0x7fbba0101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2179704938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0101210 0x7fbba0101630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009900 tx=0x7fbb9402bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:51.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 -- 172.21.15.6:0/2179704938 shutdown_connections 2024-02-26T06:27:51.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2179704938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0102dc0 0x7fbba0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2179704938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 0x7fbba0102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2179704938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0101210 0x7fbba0101630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 -- 172.21.15.6:0/2179704938 >> 172.21.15.6:0/2179704938 conn(0x7fbba00fc7a0 msgr2=0x7fbba00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:51.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 -- 172.21.15.6:0/2179704938 shutdown_connections 2024-02-26T06:27:51.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.775+0000 7fbba7e00640 1 -- 172.21.15.6:0/2179704938 wait complete. 2024-02-26T06:27:51.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.776+0000 7fbba7e00640 1 Processor -- start 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.776+0000 7fbba7e00640 1 -- start start 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0101210 0x7fbba0074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 0x7fbba0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0102dc0 0x7fbba0073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba7e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba0074210 con 0x7fbba0102dc0 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba0074380 con 0x7fbba0102400 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba01a12a0 con 0x7fbba0101210 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba5e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0102dc0 0x7fbba0073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 0x7fbba0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba5e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0102dc0 0x7fbba0073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41050/0 (socket says 172.21.15.6:41050) 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba5e00640 1 -- 172.21.15.6:0/2983113386 learned_addr learned my addr 172.21.15.6:0/2983113386 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba5400640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0101210 0x7fbba0074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba4a00640 1 -- 172.21.15.6:0/2983113386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0101210 msgr2=0x7fbba0074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba4a00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0101210 0x7fbba0074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba4a00640 1 -- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0102dc0 msgr2=0x7fbba0073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba4a00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0102dc0 0x7fbba0073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:51.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.777+0000 7fbba4a00640 1 -- 172.21.15.6:0/2983113386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb98009840 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.778+0000 7fbba4a00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 0x7fbba0073570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00ea20 tx=0x7fbb8c00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.778+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00ccc0 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.778+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb8c00ce20 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.778+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb94009560 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.778+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c018840 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.778+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba01a18d0 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.779+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba0108530 con 0x7fbba0102400 2024-02-26T06:27:51.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.781+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fbb8c0189a0 con 0x7fbba0102400 2024-02-26T06:27:51.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.784+0000 7fbb9e000640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbb880a2150 0x7fbb880a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:51.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.785+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fbb8c014070 con 0x7fbba0102400 2024-02-26T06:27:51.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.785+0000 7fbba5400640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbb880a2150 0x7fbb880a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:51.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.786+0000 7fbba5400640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbb880a2150 0x7fbb880a4610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbb9402c240 tx=0x7fbb9402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:51.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:51.786+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb8c052d20 con 0x7fbba0102400 2024-02-26T06:27:52.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.008+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbba002ce40 con 0x7fbb880a2150 2024-02-26T06:27:52.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.027+0000 7fbb9e000640 1 -- 172.21.15.6:0/2983113386 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343101 (secure 0 0 0) 0x7fbba002ce40 con 0x7fbb880a2150 2024-02-26T06:27:52.034 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:52.037 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.034+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbb880a2150 msgr2=0x7fbb880a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.034+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbb880a2150 0x7fbb880a4610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbb9402c240 tx=0x7fbb9402c7e0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.034+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 msgr2=0x7fbba0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.034+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 0x7fbba0073570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00ea20 tx=0x7fbb8c00eee0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 shutdown_connections 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba0102dc0 0x7fbba0073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fbb880a2150 0x7fbb880a4610 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba0102400 0x7fbba0073570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 --2- 172.21.15.6:0/2983113386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0101210 0x7fbba0074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 >> 172.21.15.6:0/2983113386 conn(0x7fbba00fc7a0 msgr2=0x7fbba0105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 shutdown_connections 2024-02-26T06:27:52.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.035+0000 7fbba7e00640 1 -- 172.21.15.6:0/2983113386 wait complete. 2024-02-26T06:27:52.061 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-02-26T06:27:50.170805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664603,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1616,"num_read_kb":942,"num_write":441,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181457,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383368,"kb_used_data":13416,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558456,"statfs":{"total":1151940427776,"available":1151547858944,"internally_reserved":0,"allocated":13737984,"data_stored":7456558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6775,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":79,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005615"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125767+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:26:42.125767+0000","last_peered":"2024-02-26T06:26:42.125767+0000","last_clean":"2024-02-26T06:26:42.125767+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:26:42.125767+0000","last_undegraded":"2024-02-26T06:26:42.125767+0000","last_fullsized":"2024-02-26T06:26:42.125767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121615+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:26:42.121615+0000","last_peered":"2024-02-26T06:26:42.121615+0000","last_clean":"2024-02-26T06:26:42.121615+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:26:42.121615+0000","last_undegraded":"2024-02-26T06:26:42.121615+0000","last_fullsized":"2024-02-26T06:26:42.121615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128284+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:26:42.128284+0000","last_peered":"2024-02-26T06:26:42.128284+0000","last_clean":"2024-02-26T06:26:42.128284+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:26:42.128284+0000","last_undegraded":"2024-02-26T06:26:42.128284+0000","last_fullsized":"2024-02-26T06:26:42.128284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106994+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:26:42.106994+0000","last_peered":"2024-02-26T06:26:42.106994+0000","last_clean":"2024-02-26T06:26:42.106994+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:26:42.106994+0000","last_undegraded":"2024-02-26T06:26:42.106994+0000","last_fullsized":"2024-02-26T06:26:42.106994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.102169+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:26:42.102169+0000","last_peered":"2024-02-26T06:26:42.102169+0000","last_clean":"2024-02-26T06:26:42.102169+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:26:42.102169+0000","last_undegraded":"2024-02-26T06:26:42.102169+0000","last_fullsized":"2024-02-26T06:26:42.102169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105559+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:26:42.105559+0000","last_peered":"2024-02-26T06:26:42.105559+0000","last_clean":"2024-02-26T06:26:42.105559+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:26:42.105559+0000","last_undegraded":"2024-02-26T06:26:42.105559+0000","last_fullsized":"2024-02-26T06:26:42.105559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114163+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:26:42.114163+0000","last_peered":"2024-02-26T06:26:42.114163+0000","last_clean":"2024-02-26T06:26:42.114163+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:26:42.114163+0000","last_undegraded":"2024-02-26T06:26:42.114163+0000","last_fullsized":"2024-02-26T06:26:42.114163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131277+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:26:42.131277+0000","last_peered":"2024-02-26T06:26:42.131277+0000","last_clean":"2024-02-26T06:26:42.131277+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:26:42.131277+0000","last_undegraded":"2024-02-26T06:26:42.131277+0000","last_fullsized":"2024-02-26T06:26:42.131277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114350+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:26:42.114350+0000","last_peered":"2024-02-26T06:26:42.114350+0000","last_clean":"2024-02-26T06:26:42.114350+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:26:42.114350+0000","last_undegraded":"2024-02-26T06:26:42.114350+0000","last_fullsized":"2024-02-26T06:26:42.114350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107249+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:26:42.107249+0000","last_peered":"2024-02-26T06:26:42.107249+0000","last_clean":"2024-02-26T06:26:42.107249+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:26:42.107249+0000","last_undegraded":"2024-02-26T06:26:42.107249+0000","last_fullsized":"2024-02-26T06:26:42.107249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107797+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:26:42.107797+0000","last_peered":"2024-02-26T06:26:42.107797+0000","last_clean":"2024-02-26T06:26:42.107797+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:26:42.107797+0000","last_undegraded":"2024-02-26T06:26:42.107797+0000","last_fullsized":"2024-02-26T06:26:42.107797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100577+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:26:42.100577+0000","last_peered":"2024-02-26T06:26:42.100577+0000","last_clean":"2024-02-26T06:26:42.100577+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:26:42.100577+0000","last_undegraded":"2024-02-26T06:26:42.100577+0000","last_fullsized":"2024-02-26T06:26:42.100577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123475+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:26:42.123475+0000","last_peered":"2024-02-26T06:26:42.123475+0000","last_clean":"2024-02-26T06:26:42.123475+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:26:42.123475+0000","last_undegraded":"2024-02-26T06:26:42.123475+0000","last_fullsized":"2024-02-26T06:26:42.123475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127750+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:26:42.127750+0000","last_peered":"2024-02-26T06:26:42.127750+0000","last_clean":"2024-02-26T06:26:42.127750+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:26:42.127750+0000","last_undegraded":"2024-02-26T06:26:42.127750+0000","last_fullsized":"2024-02-26T06:26:42.127750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124332+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:26:42.124332+0000","last_peered":"2024-02-26T06:26:42.124332+0000","last_clean":"2024-02-26T06:26:42.124332+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:26:42.124332+0000","last_undegraded":"2024-02-26T06:26:42.124332+0000","last_fullsized":"2024-02-26T06:26:42.124332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129391+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:26:42.129391+0000","last_peered":"2024-02-26T06:26:42.129391+0000","last_clean":"2024-02-26T06:26:42.129391+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:26:42.129391+0000","last_undegraded":"2024-02-26T06:26:42.129391+0000","last_fullsized":"2024-02-26T06:26:42.129391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125804+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:26:42.125804+0000","last_peered":"2024-02-26T06:26:42.125804+0000","last_clean":"2024-02-26T06:26:42.125804+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:26:42.125804+0000","last_undegraded":"2024-02-26T06:26:42.125804+0000","last_fullsized":"2024-02-26T06:26:42.125804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.109782+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:26:42.109782+0000","last_peered":"2024-02-26T06:26:42.109782+0000","last_clean":"2024-02-26T06:26:42.109782+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:26:42.109782+0000","last_undegraded":"2024-02-26T06:26:42.109782+0000","last_fullsized":"2024-02-26T06:26:42.109782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125127+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:26:42.125127+0000","last_peered":"2024-02-26T06:26:42.125127+0000","last_clean":"2024-02-26T06:26:42.125127+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:26:42.125127+0000","last_undegraded":"2024-02-26T06:26:42.125127+0000","last_fullsized":"2024-02-26T06:26:42.125127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":179,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:48.114147+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:27:48.114147+0000","last_peered":"2024-02-26T06:27:48.114147+0000","last_clean":"2024-02-26T06:27:48.114147+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:27:48.114147+0000","last_undegraded":"2024-02-26T06:27:48.114147+0000","last_fullsized":"2024-02-26T06:27:48.114147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120848+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:26:42.120848+0000","last_peered":"2024-02-26T06:26:42.120848+0000","last_clean":"2024-02-26T06:26:42.120848+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:26:42.120848+0000","last_undegraded":"2024-02-26T06:26:42.120848+0000","last_fullsized":"2024-02-26T06:26:42.120848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869485+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:26:13.869485+0000","last_peered":"2024-02-26T06:26:13.869485+0000","last_clean":"2024-02-26T06:26:13.869485+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:26:13.869485+0000","last_undegraded":"2024-02-26T06:26:13.869485+0000","last_fullsized":"2024-02-26T06:26:13.869485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128676+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:26:42.128676+0000","last_peered":"2024-02-26T06:26:42.128676+0000","last_clean":"2024-02-26T06:26:42.128676+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:26:42.128676+0000","last_undegraded":"2024-02-26T06:26:42.128676+0000","last_fullsized":"2024-02-26T06:26:42.128676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.864796+0000","last_change":"2024-02-26T06:26:13.864219+0000","last_active":"2024-02-26T06:26:13.864796+0000","last_peered":"2024-02-26T06:26:13.864796+0000","last_clean":"2024-02-26T06:26:13.864796+0000","last_became_active":"2024-02-26T06:26:13.851415+0000","last_became_peered":"2024-02-26T06:26:13.851415+0000","last_unstale":"2024-02-26T06:26:13.864796+0000","last_undegraded":"2024-02-26T06:26:13.864796+0000","last_fullsized":"2024-02-26T06:26:13.864796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:46:36.684794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125570+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:26:42.125570+0000","last_peered":"2024-02-26T06:26:42.125570+0000","last_clean":"2024-02-26T06:26:42.125570+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:26:42.125570+0000","last_undegraded":"2024-02-26T06:26:42.125570+0000","last_fullsized":"2024-02-26T06:26:42.125570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.874338+0000","last_change":"2024-02-26T06:26:13.873833+0000","last_active":"2024-02-26T06:26:13.874338+0000","last_peered":"2024-02-26T06:26:13.874338+0000","last_clean":"2024-02-26T06:26:13.874338+0000","last_became_active":"2024-02-26T06:26:13.852310+0000","last_became_peered":"2024-02-26T06:26:13.852310+0000","last_unstale":"2024-02-26T06:26:13.874338+0000","last_undegraded":"2024-02-26T06:26:13.874338+0000","last_fullsized":"2024-02-26T06:26:13.874338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:20.887692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.117029+0000","last_change":"2024-02-26T06:26:13.864818+0000","last_active":"2024-02-26T06:26:42.117029+0000","last_peered":"2024-02-26T06:26:42.117029+0000","last_clean":"2024-02-26T06:26:42.117029+0000","last_became_active":"2024-02-26T06:26:13.863821+0000","last_became_peered":"2024-02-26T06:26:13.863821+0000","last_unstale":"2024-02-26T06:26:42.117029+0000","last_undegraded":"2024-02-26T06:26:42.117029+0000","last_fullsized":"2024-02-26T06:26:42.117029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:38.266114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":172,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005658+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:27:49.005658+0000","last_peered":"2024-02-26T06:27:49.005658+0000","last_clean":"2024-02-26T06:27:49.005658+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:27:49.005658+0000","last_undegraded":"2024-02-26T06:27:49.005658+0000","last_fullsized":"2024-02-26T06:27:49.005658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098872+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:26:42.098872+0000","last_peered":"2024-02-26T06:26:42.098872+0000","last_clean":"2024-02-26T06:26:42.098872+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:26:42.098872+0000","last_undegraded":"2024-02-26T06:26:42.098872+0000","last_fullsized":"2024-02-26T06:26:42.098872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851763+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:26:13.851763+0000","last_peered":"2024-02-26T06:26:13.851763+0000","last_clean":"2024-02-26T06:26:13.851763+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:26:13.851763+0000","last_undegraded":"2024-02-26T06:26:13.851763+0000","last_fullsized":"2024-02-26T06:26:13.851763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131161+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:26:42.131161+0000","last_peered":"2024-02-26T06:26:42.131161+0000","last_clean":"2024-02-26T06:26:42.131161+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:26:42.131161+0000","last_undegraded":"2024-02-26T06:26:42.131161+0000","last_fullsized":"2024-02-26T06:26:42.131161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.858385+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:26:13.858385+0000","last_peered":"2024-02-26T06:26:13.858385+0000","last_clean":"2024-02-26T06:26:13.858385+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:26:13.858385+0000","last_undegraded":"2024-02-26T06:26:13.858385+0000","last_fullsized":"2024-02-26T06:26:13.858385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128140+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:26:42.128140+0000","last_peered":"2024-02-26T06:26:42.128140+0000","last_clean":"2024-02-26T06:26:42.128140+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:26:42.128140+0000","last_undegraded":"2024-02-26T06:26:42.128140+0000","last_fullsized":"2024-02-26T06:26:42.128140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.850787+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:26:13.850787+0000","last_peered":"2024-02-26T06:26:13.850787+0000","last_clean":"2024-02-26T06:26:13.850787+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:26:13.850787+0000","last_undegraded":"2024-02-26T06:26:13.850787+0000","last_fullsized":"2024-02-26T06:26:13.850787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131708+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:26:42.131708+0000","last_peered":"2024-02-26T06:26:42.131708+0000","last_clean":"2024-02-26T06:26:42.131708+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:26:42.131708+0000","last_undegraded":"2024-02-26T06:26:42.131708+0000","last_fullsized":"2024-02-26T06:26:42.131708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":315,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005095+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:27:49.005095+0000","last_peered":"2024-02-26T06:27:49.005095+0000","last_clean":"2024-02-26T06:27:49.005095+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:27:49.005095+0000","last_undegraded":"2024-02-26T06:27:49.005095+0000","last_fullsized":"2024-02-26T06:27:49.005095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":290,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.115755+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:26:42.115755+0000","last_peered":"2024-02-26T06:26:42.115755+0000","last_clean":"2024-02-26T06:26:42.115755+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:26:42.115755+0000","last_undegraded":"2024-02-26T06:26:42.115755+0000","last_fullsized":"2024-02-26T06:26:42.115755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848306+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:26:13.848306+0000","last_peered":"2024-02-26T06:26:13.848306+0000","last_clean":"2024-02-26T06:26:13.848306+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:26:13.848306+0000","last_undegraded":"2024-02-26T06:26:13.848306+0000","last_fullsized":"2024-02-26T06:26:13.848306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128679+0000","last_change":"2024-02-26T06:26:13.858594+0000","last_active":"2024-02-26T06:26:42.128679+0000","last_peered":"2024-02-26T06:26:42.128679+0000","last_clean":"2024-02-26T06:26:42.128679+0000","last_became_active":"2024-02-26T06:26:13.852999+0000","last_became_peered":"2024-02-26T06:26:13.852999+0000","last_unstale":"2024-02-26T06:26:42.128679+0000","last_undegraded":"2024-02-26T06:26:42.128679+0000","last_fullsized":"2024-02-26T06:26:42.128679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:17:51.940824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851822+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:26:13.851822+0000","last_peered":"2024-02-26T06:26:13.851822+0000","last_clean":"2024-02-26T06:26:13.851822+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:26:13.851822+0000","last_undegraded":"2024-02-26T06:26:13.851822+0000","last_fullsized":"2024-02-26T06:26:13.851822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125874+0000","last_change":"2024-02-26T06:26:13.863050+0000","last_active":"2024-02-26T06:26:42.125874+0000","last_peered":"2024-02-26T06:26:42.125874+0000","last_clean":"2024-02-26T06:26:42.125874+0000","last_became_active":"2024-02-26T06:26:13.862635+0000","last_became_peered":"2024-02-26T06:26:13.862635+0000","last_unstale":"2024-02-26T06:26:42.125874+0000","last_undegraded":"2024-02-26T06:26:42.125874+0000","last_fullsized":"2024-02-26T06:26:42.125874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:38:45.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":360,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005625+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:27:49.005625+0000","last_peered":"2024-02-26T06:27:49.005625+0000","last_clean":"2024-02-26T06:27:49.005625+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:27:49.005625+0000","last_undegraded":"2024-02-26T06:27:49.005625+0000","last_fullsized":"2024-02-26T06:27:49.005625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66346,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":301,"num_read_kb":216,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100074+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:26:42.100074+0000","last_peered":"2024-02-26T06:26:42.100074+0000","last_clean":"2024-02-26T06:26:42.100074+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:26:42.100074+0000","last_undegraded":"2024-02-26T06:26:42.100074+0000","last_fullsized":"2024-02-26T06:26:42.100074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848123+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:26:13.848123+0000","last_peered":"2024-02-26T06:26:13.848123+0000","last_clean":"2024-02-26T06:26:13.848123+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:26:13.848123+0000","last_undegraded":"2024-02-26T06:26:13.848123+0000","last_fullsized":"2024-02-26T06:26:13.848123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131830+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:26:42.131830+0000","last_peered":"2024-02-26T06:26:42.131830+0000","last_clean":"2024-02-26T06:26:42.131830+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:26:42.131830+0000","last_undegraded":"2024-02-26T06:26:42.131830+0000","last_fullsized":"2024-02-26T06:26:42.131830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851163+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:26:13.851163+0000","last_peered":"2024-02-26T06:26:13.851163+0000","last_clean":"2024-02-26T06:26:13.851163+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:26:13.851163+0000","last_undegraded":"2024-02-26T06:26:13.851163+0000","last_fullsized":"2024-02-26T06:26:13.851163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112311+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:26:42.112311+0000","last_peered":"2024-02-26T06:26:42.112311+0000","last_clean":"2024-02-26T06:26:42.112311+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:26:42.112311+0000","last_undegraded":"2024-02-26T06:26:42.112311+0000","last_fullsized":"2024-02-26T06:26:42.112311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'8","reported_seq":177,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:48.128168+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:27:48.128168+0000","last_peered":"2024-02-26T06:27:48.128168+0000","last_clean":"2024-02-26T06:27:48.128168+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:27:48.128168+0000","last_undegraded":"2024-02-26T06:27:48.128168+0000","last_fullsized":"2024-02-26T06:27:48.128168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113095+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:26:42.113095+0000","last_peered":"2024-02-26T06:26:42.113095+0000","last_clean":"2024-02-26T06:26:42.113095+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:26:42.113095+0000","last_undegraded":"2024-02-26T06:26:42.113095+0000","last_fullsized":"2024-02-26T06:26:42.113095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844759+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:26:13.844759+0000","last_peered":"2024-02-26T06:26:13.844759+0000","last_clean":"2024-02-26T06:26:13.844759+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:26:13.844759+0000","last_undegraded":"2024-02-26T06:26:13.844759+0000","last_fullsized":"2024-02-26T06:26:13.844759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113707+0000","last_change":"2024-02-26T06:26:13.862455+0000","last_active":"2024-02-26T06:26:42.113707+0000","last_peered":"2024-02-26T06:26:42.113707+0000","last_clean":"2024-02-26T06:26:42.113707+0000","last_became_active":"2024-02-26T06:26:13.861385+0000","last_became_peered":"2024-02-26T06:26:13.861385+0000","last_unstale":"2024-02-26T06:26:42.113707+0000","last_undegraded":"2024-02-26T06:26:42.113707+0000","last_fullsized":"2024-02-26T06:26:42.113707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:42:41.345104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869092+0000","last_change":"2024-02-26T06:26:13.868540+0000","last_active":"2024-02-26T06:26:13.869092+0000","last_peered":"2024-02-26T06:26:13.869092+0000","last_clean":"2024-02-26T06:26:13.869092+0000","last_became_active":"2024-02-26T06:26:13.852149+0000","last_became_peered":"2024-02-26T06:26:13.852149+0000","last_unstale":"2024-02-26T06:26:13.869092+0000","last_undegraded":"2024-02-26T06:26:13.869092+0000","last_fullsized":"2024-02-26T06:26:13.869092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:14:30.745782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101726+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:26:42.101726+0000","last_peered":"2024-02-26T06:26:42.101726+0000","last_clean":"2024-02-26T06:26:42.101726+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:26:42.101726+0000","last_undegraded":"2024-02-26T06:26:42.101726+0000","last_fullsized":"2024-02-26T06:26:42.101726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.860103+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:26:13.860103+0000","last_peered":"2024-02-26T06:26:13.860103+0000","last_clean":"2024-02-26T06:26:13.860103+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:26:13.860103+0000","last_undegraded":"2024-02-26T06:26:13.860103+0000","last_fullsized":"2024-02-26T06:26:13.860103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105524+0000","last_change":"2024-02-26T06:26:13.872244+0000","last_active":"2024-02-26T06:26:42.105524+0000","last_peered":"2024-02-26T06:26:42.105524+0000","last_clean":"2024-02-26T06:26:42.105524+0000","last_became_active":"2024-02-26T06:26:13.866094+0000","last_became_peered":"2024-02-26T06:26:13.866094+0000","last_unstale":"2024-02-26T06:26:42.105524+0000","last_undegraded":"2024-02-26T06:26:42.105524+0000","last_fullsized":"2024-02-26T06:26:42.105524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:56:52.301479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":167,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:44.106988+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:27:44.106988+0000","last_peered":"2024-02-26T06:27:44.106988+0000","last_clean":"2024-02-26T06:27:44.106988+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:27:44.106988+0000","last_undegraded":"2024-02-26T06:27:44.106988+0000","last_fullsized":"2024-02-26T06:27:44.106988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125601+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:26:42.125601+0000","last_peered":"2024-02-26T06:26:42.125601+0000","last_clean":"2024-02-26T06:26:42.125601+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:26:42.125601+0000","last_undegraded":"2024-02-26T06:26:42.125601+0000","last_fullsized":"2024-02-26T06:26:42.125601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.094140+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:26:42.094140+0000","last_peered":"2024-02-26T06:26:42.094140+0000","last_clean":"2024-02-26T06:26:42.094140+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:26:42.094140+0000","last_undegraded":"2024-02-26T06:26:42.094140+0000","last_fullsized":"2024-02-26T06:26:42.094140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":168,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:44.116835+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:27:44.116835+0000","last_peered":"2024-02-26T06:27:44.116835+0000","last_clean":"2024-02-26T06:27:44.116835+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:27:44.116835+0000","last_undegraded":"2024-02-26T06:27:44.116835+0000","last_fullsized":"2024-02-26T06:27:44.116835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130696+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:26:42.130696+0000","last_peered":"2024-02-26T06:26:42.130696+0000","last_clean":"2024-02-26T06:26:42.130696+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:26:42.130696+0000","last_undegraded":"2024-02-26T06:26:42.130696+0000","last_fullsized":"2024-02-26T06:26:42.130696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849961+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:26:13.849961+0000","last_peered":"2024-02-26T06:26:13.849961+0000","last_clean":"2024-02-26T06:26:13.849961+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:26:13.849961+0000","last_undegraded":"2024-02-26T06:26:13.849961+0000","last_fullsized":"2024-02-26T06:26:13.849961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846775+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:26:13.846775+0000","last_peered":"2024-02-26T06:26:13.846775+0000","last_clean":"2024-02-26T06:26:13.846775+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:26:13.846775+0000","last_undegraded":"2024-02-26T06:26:13.846775+0000","last_fullsized":"2024-02-26T06:26:13.846775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121139+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:26:42.121139+0000","last_peered":"2024-02-26T06:26:42.121139+0000","last_clean":"2024-02-26T06:26:42.121139+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:26:42.121139+0000","last_undegraded":"2024-02-26T06:26:42.121139+0000","last_fullsized":"2024-02-26T06:26:42.121139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846885+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:26:13.846885+0000","last_peered":"2024-02-26T06:26:13.846885+0000","last_clean":"2024-02-26T06:26:13.846885+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:26:13.846885+0000","last_undegraded":"2024-02-26T06:26:13.846885+0000","last_fullsized":"2024-02-26T06:26:13.846885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.845086+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:26:13.845086+0000","last_peered":"2024-02-26T06:26:13.845086+0000","last_clean":"2024-02-26T06:26:13.845086+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:26:13.845086+0000","last_undegraded":"2024-02-26T06:26:13.845086+0000","last_fullsized":"2024-02-26T06:26:13.845086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123071+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:26:42.123071+0000","last_peered":"2024-02-26T06:26:42.123071+0000","last_clean":"2024-02-26T06:26:42.123071+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:26:42.123071+0000","last_undegraded":"2024-02-26T06:26:42.123071+0000","last_fullsized":"2024-02-26T06:26:42.123071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:45.258314+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:27:45.258314+0000","last_peered":"2024-02-26T06:27:45.258314+0000","last_clean":"2024-02-26T06:27:45.258314+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:27:45.258314+0000","last_undegraded":"2024-02-26T06:27:45.258314+0000","last_fullsized":"2024-02-26T06:27:45.258314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126193+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:26:42.126193+0000","last_peered":"2024-02-26T06:26:42.126193+0000","last_clean":"2024-02-26T06:26:42.126193+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:26:42.126193+0000","last_undegraded":"2024-02-26T06:26:42.126193+0000","last_fullsized":"2024-02-26T06:26:42.126193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849969+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:26:13.849969+0000","last_peered":"2024-02-26T06:26:13.849969+0000","last_clean":"2024-02-26T06:26:13.849969+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:26:13.849969+0000","last_undegraded":"2024-02-26T06:26:13.849969+0000","last_fullsized":"2024-02-26T06:26:13.849969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128571+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:26:42.128571+0000","last_peered":"2024-02-26T06:26:42.128571+0000","last_clean":"2024-02-26T06:26:42.128571+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:26:42.128571+0000","last_undegraded":"2024-02-26T06:26:42.128571+0000","last_fullsized":"2024-02-26T06:26:42.128571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.847127+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:26:13.847127+0000","last_peered":"2024-02-26T06:26:13.847127+0000","last_clean":"2024-02-26T06:26:13.847127+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:26:13.847127+0000","last_undegraded":"2024-02-26T06:26:13.847127+0000","last_fullsized":"2024-02-26T06:26:13.847127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098805+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:26:42.098805+0000","last_peered":"2024-02-26T06:26:42.098805+0000","last_clean":"2024-02-26T06:26:42.098805+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:26:42.098805+0000","last_undegraded":"2024-02-26T06:26:42.098805+0000","last_fullsized":"2024-02-26T06:26:42.098805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.866330+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:26:13.866330+0000","last_peered":"2024-02-26T06:26:13.866330+0000","last_clean":"2024-02-26T06:26:13.866330+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:26:13.866330+0000","last_undegraded":"2024-02-26T06:26:13.866330+0000","last_fullsized":"2024-02-26T06:26:13.866330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125635+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:26:42.125635+0000","last_peered":"2024-02-26T06:26:42.125635+0000","last_clean":"2024-02-26T06:26:42.125635+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:26:42.125635+0000","last_undegraded":"2024-02-26T06:26:42.125635+0000","last_fullsized":"2024-02-26T06:26:42.125635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848328+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:26:13.848328+0000","last_peered":"2024-02-26T06:26:13.848328+0000","last_clean":"2024-02-26T06:26:13.848328+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:26:13.848328+0000","last_undegraded":"2024-02-26T06:26:13.848328+0000","last_fullsized":"2024-02-26T06:26:13.848328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129822+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:26:42.129822+0000","last_peered":"2024-02-26T06:26:42.129822+0000","last_clean":"2024-02-26T06:26:42.129822+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:26:42.129822+0000","last_undegraded":"2024-02-26T06:26:42.129822+0000","last_fullsized":"2024-02-26T06:26:42.129822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844858+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:26:13.844858+0000","last_peered":"2024-02-26T06:26:13.844858+0000","last_clean":"2024-02-26T06:26:13.844858+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:26:13.844858+0000","last_undegraded":"2024-02-26T06:26:13.844858+0000","last_fullsized":"2024-02-26T06:26:13.844858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112189+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:26:42.112189+0000","last_peered":"2024-02-26T06:26:42.112189+0000","last_clean":"2024-02-26T06:26:42.112189+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:26:42.112189+0000","last_undegraded":"2024-02-26T06:26:42.112189+0000","last_fullsized":"2024-02-26T06:26:42.112189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":170,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:45.000131+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:27:45.000131+0000","last_peered":"2024-02-26T06:27:45.000131+0000","last_clean":"2024-02-26T06:27:45.000131+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:27:45.000131+0000","last_undegraded":"2024-02-26T06:27:45.000131+0000","last_fullsized":"2024-02-26T06:27:45.000131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1480,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107689+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:26:42.107689+0000","last_peered":"2024-02-26T06:26:42.107689+0000","last_clean":"2024-02-26T06:26:42.107689+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:26:42.107689+0000","last_undegraded":"2024-02-26T06:26:42.107689+0000","last_fullsized":"2024-02-26T06:26:42.107689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851673+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:26:13.851673+0000","last_peered":"2024-02-26T06:26:13.851673+0000","last_clean":"2024-02-26T06:26:13.851673+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:26:13.851673+0000","last_undegraded":"2024-02-26T06:26:13.851673+0000","last_fullsized":"2024-02-26T06:26:13.851673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.111993+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:26:42.111993+0000","last_peered":"2024-02-26T06:26:42.111993+0000","last_clean":"2024-02-26T06:26:42.111993+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:26:42.111993+0000","last_undegraded":"2024-02-26T06:26:42.111993+0000","last_fullsized":"2024-02-26T06:26:42.111993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.879538+0000","last_change":"2024-02-26T06:26:13.878533+0000","last_active":"2024-02-26T06:26:13.879538+0000","last_peered":"2024-02-26T06:26:13.879538+0000","last_clean":"2024-02-26T06:26:13.879538+0000","last_became_active":"2024-02-26T06:26:13.866478+0000","last_became_peered":"2024-02-26T06:26:13.866478+0000","last_unstale":"2024-02-26T06:26:13.879538+0000","last_undegraded":"2024-02-26T06:26:13.879538+0000","last_fullsized":"2024-02-26T06:26:13.879538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:00:23.896436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099320+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:26:42.099320+0000","last_peered":"2024-02-26T06:26:42.099320+0000","last_clean":"2024-02-26T06:26:42.099320+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:26:42.099320+0000","last_undegraded":"2024-02-26T06:26:42.099320+0000","last_fullsized":"2024-02-26T06:26:42.099320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100022+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:26:42.100022+0000","last_peered":"2024-02-26T06:26:42.100022+0000","last_clean":"2024-02-26T06:26:42.100022+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:26:42.100022+0000","last_undegraded":"2024-02-26T06:26:42.100022+0000","last_fullsized":"2024-02-26T06:26:42.100022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.108252+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:26:42.108252+0000","last_peered":"2024-02-26T06:26:42.108252+0000","last_clean":"2024-02-26T06:26:42.108252+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:26:42.108252+0000","last_undegraded":"2024-02-26T06:26:42.108252+0000","last_fullsized":"2024-02-26T06:26:42.108252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124761+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:26:42.124761+0000","last_peered":"2024-02-26T06:26:42.124761+0000","last_clean":"2024-02-26T06:26:42.124761+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:26:42.124761+0000","last_undegraded":"2024-02-26T06:26:42.124761+0000","last_fullsized":"2024-02-26T06:26:42.124761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127390+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:26:42.127390+0000","last_peered":"2024-02-26T06:26:42.127390+0000","last_clean":"2024-02-26T06:26:42.127390+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:26:42.127390+0000","last_undegraded":"2024-02-26T06:26:42.127390+0000","last_fullsized":"2024-02-26T06:26:42.127390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126867+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:26:42.126867+0000","last_peered":"2024-02-26T06:26:42.126867+0000","last_clean":"2024-02-26T06:26:42.126867+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:26:42.126867+0000","last_undegraded":"2024-02-26T06:26:42.126867+0000","last_fullsized":"2024-02-26T06:26:42.126867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.096844+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:26:42.096844+0000","last_peered":"2024-02-26T06:26:42.096844+0000","last_clean":"2024-02-26T06:26:42.096844+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:26:42.096844+0000","last_undegraded":"2024-02-26T06:26:42.096844+0000","last_fullsized":"2024-02-26T06:26:42.096844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106293+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:26:42.106293+0000","last_peered":"2024-02-26T06:26:42.106293+0000","last_clean":"2024-02-26T06:26:42.106293+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:26:42.106293+0000","last_undegraded":"2024-02-26T06:26:42.106293+0000","last_fullsized":"2024-02-26T06:26:42.106293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112460+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:26:42.112460+0000","last_peered":"2024-02-26T06:26:42.112460+0000","last_clean":"2024-02-26T06:26:42.112460+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:26:42.112460+0000","last_undegraded":"2024-02-26T06:26:42.112460+0000","last_fullsized":"2024-02-26T06:26:42.112460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130712+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:26:42.130712+0000","last_peered":"2024-02-26T06:26:42.130712+0000","last_clean":"2024-02-26T06:26:42.130712+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:26:42.130712+0000","last_undegraded":"2024-02-26T06:26:42.130712+0000","last_fullsized":"2024-02-26T06:26:42.130712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125683+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:26:42.125683+0000","last_peered":"2024-02-26T06:26:42.125683+0000","last_clean":"2024-02-26T06:26:42.125683+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:26:42.125683+0000","last_undegraded":"2024-02-26T06:26:42.125683+0000","last_fullsized":"2024-02-26T06:26:42.125683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131704+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:26:42.131704+0000","last_peered":"2024-02-26T06:26:42.131704+0000","last_clean":"2024-02-26T06:26:42.131704+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:26:42.131704+0000","last_undegraded":"2024-02-26T06:26:42.131704+0000","last_fullsized":"2024-02-26T06:26:42.131704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129590+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:26:42.129590+0000","last_peered":"2024-02-26T06:26:42.129590+0000","last_clean":"2024-02-26T06:26:42.129590+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:26:42.129590+0000","last_undegraded":"2024-02-26T06:26:42.129590+0000","last_fullsized":"2024-02-26T06:26:42.129590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100819+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:26:42.100819+0000","last_peered":"2024-02-26T06:26:42.100819+0000","last_clean":"2024-02-26T06:26:42.100819+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:26:42.100819+0000","last_undegraded":"2024-02-26T06:26:42.100819+0000","last_fullsized":"2024-02-26T06:26:42.100819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74001,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1480,"num_read_kb":823,"num_write":258,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":216239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94699999999999995,"5min":1.113,"15min":1.113},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.748}]},{"osd":1,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59699999999999998,"15min":0.56699999999999995},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.36499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66000000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.51900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.751,"15min":0.751},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.71299999999999997,"5min":0.874,"15min":0.874},"last":0.51800000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.84999999999999998},"min":{"1min":0.56299999999999994,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.64600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":1.385,"15min":1.385},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.57999999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.623},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69699999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997}]},{"osd":9,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.90100000000000002,"15min":0.90000000000000002},"min":{"1min":0.67400000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.179,"5min":1.238,"15min":1.238},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.95099999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.151,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.58299999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":1.044,"15min":1.044},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.73099999999999998},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.93500000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71799999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1115232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":0.93700000000000006},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.4740000000000002,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45500000000000002,"15min":0.46300000000000002},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.373},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.627,"5min":1.093,"15min":1.093},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.752,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51600000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.167,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0920000000000001,"5min":1.458,"15min":1.458},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.52500000000000002,"15min":0.54300000000000004},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.56299999999999994,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.69499999999999995,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76900000000000002,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81100000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.73999999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82499999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60599999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.004,"15min":1.034},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1279999999999999,"5min":1.256,"15min":1.256},"last":0.98699999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77100000000000002,"15min":0.78000000000000003},"min":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.629}]}]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.367,"15min":1.419},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.8979999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.3700000000000001,"15min":1.429},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7889999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.238}]},{"osd":1,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.1020000000000001,"15min":1.1759999999999999},"min":{"1min":0.70599999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.266,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.675},{"interface":"front","average":{"1min":1.117,"5min":1.2090000000000001,"15min":1.292},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.393,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1539999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4670000000000001,"15min":1.5429999999999999},"min":{"1min":0.69799999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8759999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3320000000000001,"15min":1.3919999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0209999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.468}]},{"osd":3,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.613,"15min":1.7350000000000001},"min":{"1min":0.749,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1400000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.736},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.7410000000000001,"15min":1.839},"min":{"1min":0.89800000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1869999999999998,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":2.238}]},{"osd":4,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7470000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1859999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":2.2669999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.766,"15min":1.9159999999999999},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":2.665}]},{"osd":5,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.7709999999999999,"5min":2.012,"15min":2.0699999999999998},"min":{"1min":1.347,"5min":0.879,"15min":0.879},"max":{"1min":2.298,"5min":14.581,"15min":14.581},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.708,"5min":2.3500000000000001,"15min":2.7480000000000002},"min":{"1min":1.268,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3650000000000002,"5min":18.216000000000001,"15min":18.216000000000001},"last":2.3170000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.117,"15min":2.2599999999999998},"min":{"1min":1.2749999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.4199999999999999,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":3.4209999999999998},{"interface":"front","average":{"1min":1.8089999999999999,"5min":2.0339999999999998,"15min":2.194},"min":{"1min":1.167,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.5899999999999999,"5min":3.044,"15min":3.044},"last":3.5619999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.9339999999999999,"15min":2.0619999999999998},"min":{"1min":1.071,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.2269999999999999,"5min":3.036,"15min":3.036},"last":3.0870000000000002},{"interface":"front","average":{"1min":1.875,"5min":2.0510000000000002,"15min":2.1669999999999998},"min":{"1min":1.1499999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.4870000000000001,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":2.1230000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":2.0529999999999999,"5min":2.1589999999999998,"15min":2.2949999999999999},"min":{"1min":1.3859999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"max":{"1min":2.6160000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.1970000000000001},{"interface":"front","average":{"1min":1.889,"5min":2.0249999999999999,"15min":2.1579999999999999},"min":{"1min":1.359,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.492,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":2.355}]},{"osd":10,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":1.895,"5min":1.9390000000000001,"15min":1.9590000000000001},"min":{"1min":1.0389999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.4609999999999999,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":2.9630000000000001},{"interface":"front","average":{"1min":2.0939999999999999,"5min":2.1669999999999998,"15min":2.1949999999999998},"min":{"1min":1.089,"5min":1.089,"15min":1.089},"max":{"1min":2.6629999999999998,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":2.1619999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.44500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.505,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3069999999999999,"5min":1.478,"15min":1.478},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.47699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.752}]},{"osd":2,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.442},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45800000000000002,"15min":0.442},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.77400000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.55200000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1379999999999999,"5min":1.216,"15min":1.216},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1919999999999999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90000000000000002,"15min":0.92200000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.373,"15min":1.373},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.67700000000000005},"min":{"1min":0.56499999999999995,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64300000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91500000000000004,"15min":0.91800000000000004},"min":{"1min":0.78100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.284,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.97499999999999998,"15min":1.0069999999999999},"min":{"1min":0.72899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.079}]},{"osd":10,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.73999999999999999},"min":{"1min":0.60499999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0329999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.55400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.59599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.48799999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70699999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1049901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.40600000000000003},"min":{"1min":0.32000000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.52200000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.46300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.879,"15min":0.879},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.1240000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.497,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.495},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.76700000000000002,"5min":0.877,"15min":0.877},"last":0.79800000000000004}]},{"osd":6,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.90700000000000003,"15min":0.86899999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94999999999999996,"5min":6.9900000000000002,"15min":6.9900000000000002},"last":0.88500000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79700000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78300000000000003}]},{"osd":9,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":0.99199999999999999},"min":{"1min":0.73099999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.038},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0649999999999999,"15min":1.038},"min":{"1min":0.66800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1839999999999999,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":0.91900000000000004}]},{"osd":10,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73899999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.66100000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.90000000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82899999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.204,"15min":1.1970000000000001},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.923,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3080000000000001,"15min":1.3069999999999999},"min":{"1min":0.55900000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1749999999999998,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.1180000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0700000000000001,"15min":1.036},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6870000000000001,"5min":1.867,"15min":1.867},"last":1.073},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.1519999999999999,"15min":1.117},"min":{"1min":0.69099999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.4750000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.347}]},{"osd":2,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.23,"15min":1.1559999999999999},"min":{"1min":0.94199999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.27},{"interface":"front","average":{"1min":1.331,"5min":1.3009999999999999,"15min":1.2370000000000001},"min":{"1min":0.65800000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.2530000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.488}]},{"osd":3,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.579,"15min":1.5369999999999999},"min":{"1min":0.82499999999999996,"5min":0.501,"15min":0.501},"max":{"1min":2.5830000000000002,"5min":3.113,"15min":3.113},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.625,"15min":1.647},"min":{"1min":0.71199999999999997,"5min":0.496,"15min":0.496},"max":{"1min":3.294,"5min":3.294,"15min":3.294},"last":2.0259999999999998}]},{"osd":4,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.381,"15min":1.359},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5740000000000001,"15min":1.514},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.492,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.6559999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5640000000000001,"15min":1.613},"min":{"1min":0.95699999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7210000000000001,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":2.141},{"interface":"front","average":{"1min":1.657,"5min":1.619,"15min":1.6040000000000001},"min":{"1min":0.91600000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.786,"5min":2.798,"15min":2.798},"last":2.0990000000000002}]},{"osd":7,"last update":"Mon Feb 26 06:27:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.7410000000000001,"15min":1.571},"min":{"1min":1.0049999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.6669999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":1.784},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.7030000000000001,"15min":1.625},"min":{"1min":0.72699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1760000000000002,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":2.2050000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.827,"15min":1.7410000000000001},"min":{"1min":1.1080000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":3.3620000000000001,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9410000000000001},{"interface":"front","average":{"1min":1.867,"5min":1.7869999999999999,"15min":1.7509999999999999},"min":{"1min":1.0569999999999999,"5min":0.747,"15min":0.747},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":2.2839999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:27:28 2024","interfaces":[{"interface":"back","average":{"1min":2.004,"5min":2.04,"15min":2.234},"min":{"1min":0.94799999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":2.242},{"interface":"front","average":{"1min":2.133,"5min":2.157,"15min":2.3380000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":3.1000000000000001,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":2.0609999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9339999999999999,"5min":2.024,"15min":2.113},"min":{"1min":1.1679999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":3.2330000000000001,"5min":3.2330000000000001,"15min":3.2330000000000001},"last":2.3599999999999999},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.9670000000000001,"15min":2.008},"min":{"1min":1.042,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.1829999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":523449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.627},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.374},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0740000000000001,"5min":1.252,"15min":1.252},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Feb 26 06:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95799999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:26:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81599999999999995,"15min":0.80100000000000005},"min":{"1min":0.55200000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1639999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.34599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.43099999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55900000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.86699999999999999},"min":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.077,"5min":1.196,"15min":1.196},"last":0.879},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.90900000000000003},"min":{"1min":0.61399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.21,"15min":1.21},"last":0.96499999999999997}]},{"osd":7,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.498,"15min":0.495},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67400000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46600000000000003}]},{"osd":8,"last update":"Mon Feb 26 06:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.622}]},{"osd":9,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.84499999999999997},"min":{"1min":0.495,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.254,"5min":1.474,"15min":1.474},"last":0.748},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.92900000000000005},"min":{"1min":0.69599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.92700000000000005}]},{"osd":10,"last update":"Mon Feb 26 06:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.496}]},{"osd":11,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66000000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":458621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.96599999999999997,"15min":0.81599999999999995},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.45900000000000002},"max":{"1min":1.379,"5min":1.569,"15min":1.569},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0069999999999999,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.40899999999999997},"max":{"1min":1.3320000000000001,"5min":3.516,"15min":3.516},"last":0.94099999999999995}]},{"osd":2,"last update":"Mon Feb 26 06:27:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0649999999999999,"15min":0.92200000000000004},"min":{"1min":0.64700000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2190000000000001,"15min":1.0109999999999999},"min":{"1min":0.89900000000000002,"5min":0.495,"15min":0.48599999999999999},"max":{"1min":1.9419999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3600000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.6519999999999999,"15min":1.6679999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.3010000000000002,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.113},{"interface":"front","average":{"1min":1.782,"5min":1.794,"15min":1.796},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.3100000000000001,"5min":2.718,"15min":2.718},"last":1.486}]},{"osd":4,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.3460000000000001},"min":{"1min":0.78800000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.1160000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.139,"15min":1.133},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.621,"5min":2.069,"15min":2.069},"last":0.84799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:27:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3660000000000001,"15min":1.3220000000000001},"min":{"1min":0.77700000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.9059999999999999,"5min":2.653,"15min":2.653},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.7210000000000001,"15min":1.5600000000000001},"min":{"1min":0.97299999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.5710000000000002,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.431}]},{"osd":6,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.532,"15min":1.4530000000000001},"min":{"1min":0.745,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.46,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6890000000000001,"15min":1.575},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.383}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.7110000000000001,"15min":1.671},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.2749999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.294},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.875,"15min":1.8089999999999999},"min":{"1min":0.91200000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.552,"5min":2.911,"15min":2.911},"last":1.5229999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:27:30 2024","interfaces":[{"interface":"back","average":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.173},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.254,"5min":3.254,"15min":3.254},"last":1.4610000000000001},{"interface":"front","average":{"1min":2.0950000000000002,"5min":2.1629999999999998,"15min":2.2690000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.7999999999999998,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.5620000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0259999999999998,"5min":2.0960000000000001,"15min":2.137},"min":{"1min":1.1200000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.8620000000000001,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.673,"15min":1.7270000000000001},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":2.2290000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.196}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.8600000000000001,"15min":1.8700000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4180000000000001,"5min":2.524,"15min":2.524},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6739999999999999,"15min":1.6819999999999999},"min":{"1min":0.89500000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.4500000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.403}]}]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":518606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72699999999999998,"15min":0.67100000000000004},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.163,"15min":1.163},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.68000000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1879999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Feb 26 06:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80700000000000005,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75600000000000001,"15min":0.69899999999999995},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.94399999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.505,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.39300000000000002},"min":{"1min":0.30299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85199999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.42899999999999999},"min":{"1min":0.35399999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42199999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88600000000000001,"15min":0.88500000000000001},"min":{"1min":0.74399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.071},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.90400000000000003,"15min":0.86499999999999999},"min":{"1min":0.64600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.319,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.51300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.89000000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63800000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.86399999999999999,"15min":0.86199999999999999},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.04,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.753},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.93200000000000005,"15min":0.95799999999999996},"min":{"1min":0.67400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1850000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.0569999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.628},"min":{"1min":0.55600000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.621},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.089,"15min":1.089},"last":0.55900000000000005}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0369999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73499999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":456289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.1799999999999999,"15min":0.97899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.45,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.45},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.369,"15min":1.0549999999999999},"min":{"1min":0.71799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8440000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.6080000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.1970000000000001,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.5199999999999996,"5min":7.5199999999999996,"15min":7.5199999999999996},"last":1.224},{"interface":"front","average":{"1min":1.143,"5min":1.2010000000000001,"15min":0.997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8300000000000001,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.3129999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.252,"15min":1.077},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.748,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.262},{"interface":"front","average":{"1min":1.74,"5min":1.397,"15min":1.171},"min":{"1min":0.50800000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3760000000000003,"5min":7.3760000000000003,"15min":7.3760000000000003},"last":1.484}]},{"osd":4,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3380000000000001,"15min":1.1240000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.9450000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.028},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.4550000000000001,"15min":1.258},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0979999999999999,"5min":3.323,"15min":3.323},"last":1.411}]},{"osd":5,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.5960000000000001,"15min":1.391},"min":{"1min":0.84499999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.4039999999999999,"5min":7.4039999999999999,"15min":7.4039999999999999},"last":1.365},{"interface":"front","average":{"1min":1.51,"5min":1.5600000000000001,"15min":1.51},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4199999999999999,"5min":3.1349999999999998,"15min":5.7770000000000001},"last":1.54}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":1.829,"15min":1.7390000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2949999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.766},{"interface":"front","average":{"1min":1.556,"5min":1.671,"15min":1.4990000000000001},"min":{"1min":0.64600000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.2170000000000001,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.7490000000000001,"15min":1.7010000000000001},"min":{"1min":0.80000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2829999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.768,"5min":1.8999999999999999,"15min":1.8480000000000001},"min":{"1min":0.75,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3149999999999999,"5min":6.758,"15min":6.758},"last":1.5940000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.9870000000000001,"15min":2.145},"min":{"1min":0.72799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.6200000000000001,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.51},{"interface":"front","average":{"1min":1.847,"5min":2.0819999999999999,"15min":2.2349999999999999},"min":{"1min":0.95299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.5449999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":1.837}]},{"osd":10,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":2.0750000000000002,"15min":2.1789999999999998},"min":{"1min":0.77400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.669,"5min":2.0049999999999999,"15min":2.153},"min":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.8140000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.6870000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.929,"5min":1.8320000000000001,"15min":1.8160000000000001},"min":{"1min":1.1919999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.6549999999999998,"5min":3.085,"15min":3.085},"last":1.74},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7090000000000001,"15min":1.6910000000000001},"min":{"1min":0.79600000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.5059999999999998,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.7889999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.41199999999999998,"15min":0.38200000000000001},"min":{"1min":0.32500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.752,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.745},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.54300000000000004},"min":{"1min":0.46600000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":1.04,"15min":1.04},"last":0.5}]},{"osd":3,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78500000000000003,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.022,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":0.71499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1439999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.56899999999999995},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82699999999999996,"15min":0.79100000000000004},"min":{"1min":0.48999999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.02,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.82999999999999996,"15min":0.84399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1659999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.48299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85199999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92200000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66600000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86299999999999999,"15min":0.89500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.173,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94299999999999995,"15min":0.99299999999999999},"min":{"1min":0.69599999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.0580000000000001,"5min":1.665,"15min":1.665},"last":0.59899999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68500000000000005,"15min":0.67400000000000004},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89500000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.47099999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78200000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88400000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.81399999999999995,"15min":0.80900000000000005},"min":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.73299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1049047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73899999999999999},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1339999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.754,"5min":0.76600000000000001,"15min":0.95699999999999996},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.03,"5min":1.99,"15min":7.2249999999999996},"last":0.73599999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.21,"15min":1.21},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.5,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84499999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.54200000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.44},{"interface":"front","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.629,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.74399999999999999},"min":{"1min":0.54000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1830000000000001,"5min":1.298,"15min":1.298},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.82599999999999996,"15min":0.76600000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81499999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80500000000000005,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.46200000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81899999999999995,"15min":0.85099999999999998},"min":{"1min":0.60499999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.075,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.65700000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.109,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.196,"15min":1.196},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.55500000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":1.177,"15min":1.177},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.441,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.40600000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.502}]},{"osd":9,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93200000000000005,"15min":0.89100000000000001},"min":{"1min":0.73299999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.1910000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0129999999999999,"15min":0.996},"min":{"1min":0.79800000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.266,"5min":1.667,"15min":1.667},"last":0.76500000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.53700000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.413,"15min":1.413},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:27:52.067 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-26T06:27:52.067 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T06:27:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:51 smithi090 ceph-mon[35681]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:27:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:51 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:51 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/483501381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:52.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:51 smithi006 ceph-mon[31790]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:27:52.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:52.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:51 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:52.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:51 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/483501381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:52.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:51 smithi167 ceph-mon[36229]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:27:52.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:52.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:51 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:52.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:51 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/483501381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:52.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 -- 172.21.15.6:0/2588713819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641045b0 msgr2=0x7fcf64104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 --2- 172.21.15.6:0/2588713819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641045b0 0x7fcf64104a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009900 tx=0x7fcf5002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 -- 172.21.15.6:0/2588713819 shutdown_connections 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 --2- 172.21.15.6:0/2588713819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf64104f70 0x7fcf6410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 --2- 172.21.15.6:0/2588713819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641045b0 0x7fcf64104a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 --2- 172.21.15.6:0/2588713819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf641033c0 0x7fcf641037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 -- 172.21.15.6:0/2588713819 >> 172.21.15.6:0/2588713819 conn(0x7fcf640fe930 msgr2=0x7fcf64100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 -- 172.21.15.6:0/2588713819 shutdown_connections 2024-02-26T06:27:52.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.314+0000 7fcf6c200640 1 -- 172.21.15.6:0/2588713819 wait complete. 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.315+0000 7fcf6c200640 1 Processor -- start 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.315+0000 7fcf6c200640 1 -- start start 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6c200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 0x7fcf64199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf641045b0 0x7fcf64199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6c200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf64104f70 0x7fcf641a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6c200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf641a0ef0 con 0x7fcf641033c0 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6c200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf641a1060 con 0x7fcf64104f70 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf641a1220 con 0x7fcf641045b0 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf69800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 0x7fcf64199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf69800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 0x7fcf64199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41074/0 (socket says 172.21.15.6:41074) 2024-02-26T06:27:52.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf69800640 1 -- 172.21.15.6:0/1012890491 learned_addr learned my addr 172.21.15.6:0/1012890491 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf68e00640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf641045b0 0x7fcf64199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf6a200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf64104f70 0x7fcf641a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf69800640 1 -- 172.21.15.6:0/1012890491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf641045b0 msgr2=0x7fcf64199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.316+0000 7fcf69800640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf641045b0 0x7fcf64199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf69800640 1 -- 172.21.15.6:0/1012890491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf64104f70 msgr2=0x7fcf641a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf69800640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf64104f70 0x7fcf641a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf69800640 1 -- 172.21.15.6:0/1012890491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c009490 con 0x7fcf641033c0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf69800640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 0x7fcf64199400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800ebc0 tx=0x7fcf5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58003970 con 0x7fcf641033c0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf58004030 con 0x7fcf641033c0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58019720 con 0x7fcf641033c0 2024-02-26T06:27:52.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf50009560 con 0x7fcf641033c0 2024-02-26T06:27:52.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.317+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf641a1800 con 0x7fcf641033c0 2024-02-26T06:27:52.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.319+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6410a760 con 0x7fcf641033c0 2024-02-26T06:27:52.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.321+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fcf580044f0 con 0x7fcf641033c0 2024-02-26T06:27:52.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.322+0000 7fcf62000640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcf4c0a21d0 0x7fcf4c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.322+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fcf58014070 con 0x7fcf641033c0 2024-02-26T06:27:52.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.322+0000 7fcf68e00640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcf4c0a21d0 0x7fcf4c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.323+0000 7fcf68e00640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcf4c0a21d0 0x7fcf4c0a4690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.326+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf580976c0 con 0x7fcf641033c0 2024-02-26T06:27:52.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.541+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf6402cf20 con 0x7fcf4c0a21d0 2024-02-26T06:27:52.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.573+0000 7fcf62000640 1 -- 172.21.15.6:0/1012890491 <== mgr.14152 v2:172.21.15.6:6800/324687139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343109 (secure 0 0 0) 0x7fcf6402cf20 con 0x7fcf4c0a21d0 2024-02-26T06:27:52.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:52.586 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-02-26T06:27:52.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcf4c0a21d0 msgr2=0x7fcf4c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcf4c0a21d0 0x7fcf4c0a4690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf50005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 msgr2=0x7fcf64199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 0x7fcf64199400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800ebc0 tx=0x7fcf5800c2d0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 shutdown_connections 2024-02-26T06:27:52.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fcf4c0a21d0 0x7fcf4c0a4690 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf64104f70 0x7fcf641a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf641045b0 0x7fcf64199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 --2- 172.21.15.6:0/1012890491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf641033c0 0x7fcf64199400 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 >> 172.21.15.6:0/1012890491 conn(0x7fcf640fe930 msgr2=0x7fcf64107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 shutdown_connections 2024-02-26T06:27:52.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.581+0000 7fcf6c200640 1 -- 172.21.15.6:0/1012890491 wait complete. 2024-02-26T06:27:52.608 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-02-26T06:27:52.171459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664994,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1632,"num_read_kb":948,"num_write":445,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181457,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383380,"kb_used_data":13428,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558444,"statfs":{"total":1151940427776,"available":1151547846656,"internally_reserved":0,"allocated":13750272,"data_stored":7466406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7166,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":65,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005215"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125767+0000","last_change":"2024-02-26T06:25:18.223809+0000","last_active":"2024-02-26T06:26:42.125767+0000","last_peered":"2024-02-26T06:26:42.125767+0000","last_clean":"2024-02-26T06:26:42.125767+0000","last_became_active":"2024-02-26T06:25:18.223417+0000","last_became_peered":"2024-02-26T06:25:18.223417+0000","last_unstale":"2024-02-26T06:26:42.125767+0000","last_undegraded":"2024-02-26T06:26:42.125767+0000","last_fullsized":"2024-02-26T06:26:42.125767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:28.364047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121615+0000","last_change":"2024-02-26T06:25:18.215382+0000","last_active":"2024-02-26T06:26:42.121615+0000","last_peered":"2024-02-26T06:26:42.121615+0000","last_clean":"2024-02-26T06:26:42.121615+0000","last_became_active":"2024-02-26T06:25:18.214975+0000","last_became_peered":"2024-02-26T06:25:18.214975+0000","last_unstale":"2024-02-26T06:26:42.121615+0000","last_undegraded":"2024-02-26T06:26:42.121615+0000","last_fullsized":"2024-02-26T06:26:42.121615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:48.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128284+0000","last_change":"2024-02-26T06:25:18.220031+0000","last_active":"2024-02-26T06:26:42.128284+0000","last_peered":"2024-02-26T06:26:42.128284+0000","last_clean":"2024-02-26T06:26:42.128284+0000","last_became_active":"2024-02-26T06:25:18.218608+0000","last_became_peered":"2024-02-26T06:25:18.218608+0000","last_unstale":"2024-02-26T06:26:42.128284+0000","last_undegraded":"2024-02-26T06:26:42.128284+0000","last_fullsized":"2024-02-26T06:26:42.128284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:22:40.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106994+0000","last_change":"2024-02-26T06:25:18.203456+0000","last_active":"2024-02-26T06:26:42.106994+0000","last_peered":"2024-02-26T06:26:42.106994+0000","last_clean":"2024-02-26T06:26:42.106994+0000","last_became_active":"2024-02-26T06:25:18.202970+0000","last_became_peered":"2024-02-26T06:25:18.202970+0000","last_unstale":"2024-02-26T06:26:42.106994+0000","last_undegraded":"2024-02-26T06:26:42.106994+0000","last_fullsized":"2024-02-26T06:26:42.106994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:09:34.062714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.102169+0000","last_change":"2024-02-26T06:25:18.219145+0000","last_active":"2024-02-26T06:26:42.102169+0000","last_peered":"2024-02-26T06:26:42.102169+0000","last_clean":"2024-02-26T06:26:42.102169+0000","last_became_active":"2024-02-26T06:25:18.218579+0000","last_became_peered":"2024-02-26T06:25:18.218579+0000","last_unstale":"2024-02-26T06:26:42.102169+0000","last_undegraded":"2024-02-26T06:26:42.102169+0000","last_fullsized":"2024-02-26T06:26:42.102169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:26:18.157231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105559+0000","last_change":"2024-02-26T06:25:18.211638+0000","last_active":"2024-02-26T06:26:42.105559+0000","last_peered":"2024-02-26T06:26:42.105559+0000","last_clean":"2024-02-26T06:26:42.105559+0000","last_became_active":"2024-02-26T06:25:18.211144+0000","last_became_peered":"2024-02-26T06:25:18.211144+0000","last_unstale":"2024-02-26T06:26:42.105559+0000","last_undegraded":"2024-02-26T06:26:42.105559+0000","last_fullsized":"2024-02-26T06:26:42.105559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:27.914376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114295+0000","last_change":"2024-02-26T06:25:18.216138+0000","last_active":"2024-02-26T06:26:42.114295+0000","last_peered":"2024-02-26T06:26:42.114295+0000","last_clean":"2024-02-26T06:26:42.114295+0000","last_became_active":"2024-02-26T06:25:18.215860+0000","last_became_peered":"2024-02-26T06:25:18.215860+0000","last_unstale":"2024-02-26T06:26:42.114295+0000","last_undegraded":"2024-02-26T06:26:42.114295+0000","last_fullsized":"2024-02-26T06:26:42.114295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:41.688499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131277+0000","last_change":"2024-02-26T06:25:18.216419+0000","last_active":"2024-02-26T06:26:42.131277+0000","last_peered":"2024-02-26T06:26:42.131277+0000","last_clean":"2024-02-26T06:26:42.131277+0000","last_became_active":"2024-02-26T06:25:18.213769+0000","last_became_peered":"2024-02-26T06:25:18.213769+0000","last_unstale":"2024-02-26T06:26:42.131277+0000","last_undegraded":"2024-02-26T06:26:42.131277+0000","last_fullsized":"2024-02-26T06:26:42.131277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:55.700559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.114350+0000","last_change":"2024-02-26T06:25:18.206560+0000","last_active":"2024-02-26T06:26:42.114350+0000","last_peered":"2024-02-26T06:26:42.114350+0000","last_clean":"2024-02-26T06:26:42.114350+0000","last_became_active":"2024-02-26T06:25:18.206199+0000","last_became_peered":"2024-02-26T06:25:18.206199+0000","last_unstale":"2024-02-26T06:26:42.114350+0000","last_undegraded":"2024-02-26T06:26:42.114350+0000","last_fullsized":"2024-02-26T06:26:42.114350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:24.638509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107249+0000","last_change":"2024-02-26T06:25:18.201379+0000","last_active":"2024-02-26T06:26:42.107249+0000","last_peered":"2024-02-26T06:26:42.107249+0000","last_clean":"2024-02-26T06:26:42.107249+0000","last_became_active":"2024-02-26T06:25:18.200872+0000","last_became_peered":"2024-02-26T06:25:18.200872+0000","last_unstale":"2024-02-26T06:26:42.107249+0000","last_undegraded":"2024-02-26T06:26:42.107249+0000","last_fullsized":"2024-02-26T06:26:42.107249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:33:45.203888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107797+0000","last_change":"2024-02-26T06:25:18.204290+0000","last_active":"2024-02-26T06:26:42.107797+0000","last_peered":"2024-02-26T06:26:42.107797+0000","last_clean":"2024-02-26T06:26:42.107797+0000","last_became_active":"2024-02-26T06:25:18.203653+0000","last_became_peered":"2024-02-26T06:25:18.203653+0000","last_unstale":"2024-02-26T06:26:42.107797+0000","last_undegraded":"2024-02-26T06:26:42.107797+0000","last_fullsized":"2024-02-26T06:26:42.107797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:30:13.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100577+0000","last_change":"2024-02-26T06:25:18.205445+0000","last_active":"2024-02-26T06:26:42.100577+0000","last_peered":"2024-02-26T06:26:42.100577+0000","last_clean":"2024-02-26T06:26:42.100577+0000","last_became_active":"2024-02-26T06:25:18.205053+0000","last_became_peered":"2024-02-26T06:25:18.205053+0000","last_unstale":"2024-02-26T06:26:42.100577+0000","last_undegraded":"2024-02-26T06:26:42.100577+0000","last_fullsized":"2024-02-26T06:26:42.100577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:54.193008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123475+0000","last_change":"2024-02-26T06:25:18.187869+0000","last_active":"2024-02-26T06:26:42.123475+0000","last_peered":"2024-02-26T06:26:42.123475+0000","last_clean":"2024-02-26T06:26:42.123475+0000","last_became_active":"2024-02-26T06:25:18.186643+0000","last_became_peered":"2024-02-26T06:25:18.186643+0000","last_unstale":"2024-02-26T06:26:42.123475+0000","last_undegraded":"2024-02-26T06:26:42.123475+0000","last_fullsized":"2024-02-26T06:26:42.123475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:12.806045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127750+0000","last_change":"2024-02-26T06:25:18.229344+0000","last_active":"2024-02-26T06:26:42.127750+0000","last_peered":"2024-02-26T06:26:42.127750+0000","last_clean":"2024-02-26T06:26:42.127750+0000","last_became_active":"2024-02-26T06:25:18.228096+0000","last_became_peered":"2024-02-26T06:25:18.228096+0000","last_unstale":"2024-02-26T06:26:42.127750+0000","last_undegraded":"2024-02-26T06:26:42.127750+0000","last_fullsized":"2024-02-26T06:26:42.127750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:06:39.756388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124332+0000","last_change":"2024-02-26T06:25:18.219404+0000","last_active":"2024-02-26T06:26:42.124332+0000","last_peered":"2024-02-26T06:26:42.124332+0000","last_clean":"2024-02-26T06:26:42.124332+0000","last_became_active":"2024-02-26T06:25:18.217746+0000","last_became_peered":"2024-02-26T06:25:18.217746+0000","last_unstale":"2024-02-26T06:26:42.124332+0000","last_undegraded":"2024-02-26T06:26:42.124332+0000","last_fullsized":"2024-02-26T06:26:42.124332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:44:17.012856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129391+0000","last_change":"2024-02-26T06:25:18.220686+0000","last_active":"2024-02-26T06:26:42.129391+0000","last_peered":"2024-02-26T06:26:42.129391+0000","last_clean":"2024-02-26T06:26:42.129391+0000","last_became_active":"2024-02-26T06:25:18.220300+0000","last_became_peered":"2024-02-26T06:25:18.220300+0000","last_unstale":"2024-02-26T06:26:42.129391+0000","last_undegraded":"2024-02-26T06:26:42.129391+0000","last_fullsized":"2024-02-26T06:26:42.129391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:42:13.030810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125804+0000","last_change":"2024-02-26T06:25:18.223804+0000","last_active":"2024-02-26T06:26:42.125804+0000","last_peered":"2024-02-26T06:26:42.125804+0000","last_clean":"2024-02-26T06:26:42.125804+0000","last_became_active":"2024-02-26T06:25:18.223409+0000","last_became_peered":"2024-02-26T06:25:18.223409+0000","last_unstale":"2024-02-26T06:26:42.125804+0000","last_undegraded":"2024-02-26T06:26:42.125804+0000","last_fullsized":"2024-02-26T06:26:42.125804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:28:38.672420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.109782+0000","last_change":"2024-02-26T06:25:18.210732+0000","last_active":"2024-02-26T06:26:42.109782+0000","last_peered":"2024-02-26T06:26:42.109782+0000","last_clean":"2024-02-26T06:26:42.109782+0000","last_became_active":"2024-02-26T06:25:18.210335+0000","last_became_peered":"2024-02-26T06:25:18.210335+0000","last_unstale":"2024-02-26T06:26:42.109782+0000","last_undegraded":"2024-02-26T06:26:42.109782+0000","last_fullsized":"2024-02-26T06:26:42.109782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:58:45.324267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125127+0000","last_change":"2024-02-26T06:25:18.218883+0000","last_active":"2024-02-26T06:26:42.125127+0000","last_peered":"2024-02-26T06:26:42.125127+0000","last_clean":"2024-02-26T06:26:42.125127+0000","last_became_active":"2024-02-26T06:25:18.218483+0000","last_became_peered":"2024-02-26T06:25:18.218483+0000","last_unstale":"2024-02-26T06:26:42.125127+0000","last_undegraded":"2024-02-26T06:26:42.125127+0000","last_fullsized":"2024-02-26T06:26:42.125127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:40.990147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":179,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:48.114147+0000","last_change":"2024-02-26T06:25:17.204711+0000","last_active":"2024-02-26T06:27:48.114147+0000","last_peered":"2024-02-26T06:27:48.114147+0000","last_clean":"2024-02-26T06:27:48.114147+0000","last_became_active":"2024-02-26T06:25:17.204351+0000","last_became_peered":"2024-02-26T06:25:17.204351+0000","last_unstale":"2024-02-26T06:27:48.114147+0000","last_undegraded":"2024-02-26T06:27:48.114147+0000","last_fullsized":"2024-02-26T06:27:48.114147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:56.601749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.120848+0000","last_change":"2024-02-26T06:25:18.225974+0000","last_active":"2024-02-26T06:26:42.120848+0000","last_peered":"2024-02-26T06:26:42.120848+0000","last_clean":"2024-02-26T06:26:42.120848+0000","last_became_active":"2024-02-26T06:25:18.221894+0000","last_became_peered":"2024-02-26T06:25:18.221894+0000","last_unstale":"2024-02-26T06:26:42.120848+0000","last_undegraded":"2024-02-26T06:26:42.120848+0000","last_fullsized":"2024-02-26T06:26:42.120848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:33:43.279143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869485+0000","last_change":"2024-02-26T06:25:17.216049+0000","last_active":"2024-02-26T06:26:13.869485+0000","last_peered":"2024-02-26T06:26:13.869485+0000","last_clean":"2024-02-26T06:26:13.869485+0000","last_became_active":"2024-02-26T06:25:17.215664+0000","last_became_peered":"2024-02-26T06:25:17.215664+0000","last_unstale":"2024-02-26T06:26:13.869485+0000","last_undegraded":"2024-02-26T06:26:13.869485+0000","last_fullsized":"2024-02-26T06:26:13.869485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:04:27.228778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128676+0000","last_change":"2024-02-26T06:25:18.224956+0000","last_active":"2024-02-26T06:26:42.128676+0000","last_peered":"2024-02-26T06:26:42.128676+0000","last_clean":"2024-02-26T06:26:42.128676+0000","last_became_active":"2024-02-26T06:25:18.224477+0000","last_became_peered":"2024-02-26T06:25:18.224477+0000","last_unstale":"2024-02-26T06:26:42.128676+0000","last_undegraded":"2024-02-26T06:26:42.128676+0000","last_fullsized":"2024-02-26T06:26:42.128676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:49:19.687065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.864796+0000","last_change":"2024-02-26T06:26:13.864219+0000","last_active":"2024-02-26T06:26:13.864796+0000","last_peered":"2024-02-26T06:26:13.864796+0000","last_clean":"2024-02-26T06:26:13.864796+0000","last_became_active":"2024-02-26T06:26:13.851415+0000","last_became_peered":"2024-02-26T06:26:13.851415+0000","last_unstale":"2024-02-26T06:26:13.864796+0000","last_undegraded":"2024-02-26T06:26:13.864796+0000","last_fullsized":"2024-02-26T06:26:13.864796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:46:36.684794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125773+0000","last_change":"2024-02-26T06:25:18.208808+0000","last_active":"2024-02-26T06:26:42.125773+0000","last_peered":"2024-02-26T06:26:42.125773+0000","last_clean":"2024-02-26T06:26:42.125773+0000","last_became_active":"2024-02-26T06:25:18.208450+0000","last_became_peered":"2024-02-26T06:25:18.208450+0000","last_unstale":"2024-02-26T06:26:42.125773+0000","last_undegraded":"2024-02-26T06:26:42.125773+0000","last_fullsized":"2024-02-26T06:26:42.125773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:04:44.160180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.874338+0000","last_change":"2024-02-26T06:26:13.873833+0000","last_active":"2024-02-26T06:26:13.874338+0000","last_peered":"2024-02-26T06:26:13.874338+0000","last_clean":"2024-02-26T06:26:13.874338+0000","last_became_active":"2024-02-26T06:26:13.852310+0000","last_became_peered":"2024-02-26T06:26:13.852310+0000","last_unstale":"2024-02-26T06:26:13.874338+0000","last_undegraded":"2024-02-26T06:26:13.874338+0000","last_fullsized":"2024-02-26T06:26:13.874338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:20.887692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.117029+0000","last_change":"2024-02-26T06:26:13.864818+0000","last_active":"2024-02-26T06:26:42.117029+0000","last_peered":"2024-02-26T06:26:42.117029+0000","last_clean":"2024-02-26T06:26:42.117029+0000","last_became_active":"2024-02-26T06:26:13.863821+0000","last_became_peered":"2024-02-26T06:26:13.863821+0000","last_unstale":"2024-02-26T06:26:42.117029+0000","last_undegraded":"2024-02-26T06:26:42.117029+0000","last_fullsized":"2024-02-26T06:26:42.117029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:53:38.266114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":172,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005658+0000","last_change":"2024-02-26T06:25:17.199032+0000","last_active":"2024-02-26T06:27:49.005658+0000","last_peered":"2024-02-26T06:27:49.005658+0000","last_clean":"2024-02-26T06:27:49.005658+0000","last_became_active":"2024-02-26T06:25:17.198661+0000","last_became_peered":"2024-02-26T06:25:17.198661+0000","last_unstale":"2024-02-26T06:27:49.005658+0000","last_undegraded":"2024-02-26T06:27:49.005658+0000","last_fullsized":"2024-02-26T06:27:49.005658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:03:31.769183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098872+0000","last_change":"2024-02-26T06:25:18.213113+0000","last_active":"2024-02-26T06:26:42.098872+0000","last_peered":"2024-02-26T06:26:42.098872+0000","last_clean":"2024-02-26T06:26:42.098872+0000","last_became_active":"2024-02-26T06:25:18.212663+0000","last_became_peered":"2024-02-26T06:25:18.212663+0000","last_unstale":"2024-02-26T06:26:42.098872+0000","last_undegraded":"2024-02-26T06:26:42.098872+0000","last_fullsized":"2024-02-26T06:26:42.098872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:08.781848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851763+0000","last_change":"2024-02-26T06:25:17.210052+0000","last_active":"2024-02-26T06:26:13.851763+0000","last_peered":"2024-02-26T06:26:13.851763+0000","last_clean":"2024-02-26T06:26:13.851763+0000","last_became_active":"2024-02-26T06:25:17.209687+0000","last_became_peered":"2024-02-26T06:25:17.209687+0000","last_unstale":"2024-02-26T06:26:13.851763+0000","last_undegraded":"2024-02-26T06:26:13.851763+0000","last_fullsized":"2024-02-26T06:26:13.851763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:40:10.938527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131161+0000","last_change":"2024-02-26T06:25:18.221504+0000","last_active":"2024-02-26T06:26:42.131161+0000","last_peered":"2024-02-26T06:26:42.131161+0000","last_clean":"2024-02-26T06:26:42.131161+0000","last_became_active":"2024-02-26T06:25:18.221253+0000","last_became_peered":"2024-02-26T06:25:18.221253+0000","last_unstale":"2024-02-26T06:26:42.131161+0000","last_undegraded":"2024-02-26T06:26:42.131161+0000","last_fullsized":"2024-02-26T06:26:42.131161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:40:53.766823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.858385+0000","last_change":"2024-02-26T06:25:17.207494+0000","last_active":"2024-02-26T06:26:13.858385+0000","last_peered":"2024-02-26T06:26:13.858385+0000","last_clean":"2024-02-26T06:26:13.858385+0000","last_became_active":"2024-02-26T06:25:17.206156+0000","last_became_peered":"2024-02-26T06:25:17.206156+0000","last_unstale":"2024-02-26T06:26:13.858385+0000","last_undegraded":"2024-02-26T06:26:13.858385+0000","last_fullsized":"2024-02-26T06:26:13.858385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:30:59.461733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128140+0000","last_change":"2024-02-26T06:25:18.218153+0000","last_active":"2024-02-26T06:26:42.128140+0000","last_peered":"2024-02-26T06:26:42.128140+0000","last_clean":"2024-02-26T06:26:42.128140+0000","last_became_active":"2024-02-26T06:25:18.217716+0000","last_became_peered":"2024-02-26T06:25:18.217716+0000","last_unstale":"2024-02-26T06:26:42.128140+0000","last_undegraded":"2024-02-26T06:26:42.128140+0000","last_fullsized":"2024-02-26T06:26:42.128140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:17.231096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.850787+0000","last_change":"2024-02-26T06:25:17.196734+0000","last_active":"2024-02-26T06:26:13.850787+0000","last_peered":"2024-02-26T06:26:13.850787+0000","last_clean":"2024-02-26T06:26:13.850787+0000","last_became_active":"2024-02-26T06:25:17.196341+0000","last_became_peered":"2024-02-26T06:25:17.196341+0000","last_unstale":"2024-02-26T06:26:13.850787+0000","last_undegraded":"2024-02-26T06:26:13.850787+0000","last_fullsized":"2024-02-26T06:26:13.850787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:47:35.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131708+0000","last_change":"2024-02-26T06:25:18.222858+0000","last_active":"2024-02-26T06:26:42.131708+0000","last_peered":"2024-02-26T06:26:42.131708+0000","last_clean":"2024-02-26T06:26:42.131708+0000","last_became_active":"2024-02-26T06:25:18.222492+0000","last_became_peered":"2024-02-26T06:25:18.222492+0000","last_unstale":"2024-02-26T06:26:42.131708+0000","last_undegraded":"2024-02-26T06:26:42.131708+0000","last_fullsized":"2024-02-26T06:26:42.131708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:56:39.451401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":315,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005095+0000","last_change":"2024-02-26T06:25:17.206702+0000","last_active":"2024-02-26T06:27:49.005095+0000","last_peered":"2024-02-26T06:27:49.005095+0000","last_clean":"2024-02-26T06:27:49.005095+0000","last_became_active":"2024-02-26T06:25:17.206329+0000","last_became_peered":"2024-02-26T06:25:17.206329+0000","last_unstale":"2024-02-26T06:27:49.005095+0000","last_undegraded":"2024-02-26T06:27:49.005095+0000","last_fullsized":"2024-02-26T06:27:49.005095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:16.753450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":290,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.115755+0000","last_change":"2024-02-26T06:25:18.209825+0000","last_active":"2024-02-26T06:26:42.115755+0000","last_peered":"2024-02-26T06:26:42.115755+0000","last_clean":"2024-02-26T06:26:42.115755+0000","last_became_active":"2024-02-26T06:25:18.209435+0000","last_became_peered":"2024-02-26T06:25:18.209435+0000","last_unstale":"2024-02-26T06:26:42.115755+0000","last_undegraded":"2024-02-26T06:26:42.115755+0000","last_fullsized":"2024-02-26T06:26:42.115755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:40.739133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848306+0000","last_change":"2024-02-26T06:25:17.197898+0000","last_active":"2024-02-26T06:26:13.848306+0000","last_peered":"2024-02-26T06:26:13.848306+0000","last_clean":"2024-02-26T06:26:13.848306+0000","last_became_active":"2024-02-26T06:25:17.197520+0000","last_became_peered":"2024-02-26T06:25:17.197520+0000","last_unstale":"2024-02-26T06:26:13.848306+0000","last_undegraded":"2024-02-26T06:26:13.848306+0000","last_fullsized":"2024-02-26T06:26:13.848306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:18.609994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128679+0000","last_change":"2024-02-26T06:26:13.858594+0000","last_active":"2024-02-26T06:26:42.128679+0000","last_peered":"2024-02-26T06:26:42.128679+0000","last_clean":"2024-02-26T06:26:42.128679+0000","last_became_active":"2024-02-26T06:26:13.852999+0000","last_became_peered":"2024-02-26T06:26:13.852999+0000","last_unstale":"2024-02-26T06:26:42.128679+0000","last_undegraded":"2024-02-26T06:26:42.128679+0000","last_fullsized":"2024-02-26T06:26:42.128679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:17:51.940824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851822+0000","last_change":"2024-02-26T06:25:17.210467+0000","last_active":"2024-02-26T06:26:13.851822+0000","last_peered":"2024-02-26T06:26:13.851822+0000","last_clean":"2024-02-26T06:26:13.851822+0000","last_became_active":"2024-02-26T06:25:17.210094+0000","last_became_peered":"2024-02-26T06:25:17.210094+0000","last_unstale":"2024-02-26T06:26:13.851822+0000","last_undegraded":"2024-02-26T06:26:13.851822+0000","last_fullsized":"2024-02-26T06:26:13.851822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:57.502386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125874+0000","last_change":"2024-02-26T06:26:13.863050+0000","last_active":"2024-02-26T06:26:42.125874+0000","last_peered":"2024-02-26T06:26:42.125874+0000","last_clean":"2024-02-26T06:26:42.125874+0000","last_became_active":"2024-02-26T06:26:13.862635+0000","last_became_peered":"2024-02-26T06:26:13.862635+0000","last_unstale":"2024-02-26T06:26:42.125874+0000","last_undegraded":"2024-02-26T06:26:42.125874+0000","last_fullsized":"2024-02-26T06:26:42.125874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:38:45.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":360,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005625+0000","last_change":"2024-02-26T06:25:17.197709+0000","last_active":"2024-02-26T06:27:49.005625+0000","last_peered":"2024-02-26T06:27:49.005625+0000","last_clean":"2024-02-26T06:27:49.005625+0000","last_became_active":"2024-02-26T06:25:17.197250+0000","last_became_peered":"2024-02-26T06:25:17.197250+0000","last_unstale":"2024-02-26T06:27:49.005625+0000","last_undegraded":"2024-02-26T06:27:49.005625+0000","last_fullsized":"2024-02-26T06:27:49.005625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:58:38.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66346,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":301,"num_read_kb":216,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100074+0000","last_change":"2024-02-26T06:25:18.205508+0000","last_active":"2024-02-26T06:26:42.100074+0000","last_peered":"2024-02-26T06:26:42.100074+0000","last_clean":"2024-02-26T06:26:42.100074+0000","last_became_active":"2024-02-26T06:25:18.205150+0000","last_became_peered":"2024-02-26T06:25:18.205150+0000","last_unstale":"2024-02-26T06:26:42.100074+0000","last_undegraded":"2024-02-26T06:26:42.100074+0000","last_fullsized":"2024-02-26T06:26:42.100074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:39.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.848123+0000","last_change":"2024-02-26T06:25:17.212788+0000","last_active":"2024-02-26T06:26:13.848123+0000","last_peered":"2024-02-26T06:26:13.848123+0000","last_clean":"2024-02-26T06:26:13.848123+0000","last_became_active":"2024-02-26T06:25:17.212445+0000","last_became_peered":"2024-02-26T06:25:17.212445+0000","last_unstale":"2024-02-26T06:26:13.848123+0000","last_undegraded":"2024-02-26T06:26:13.848123+0000","last_fullsized":"2024-02-26T06:26:13.848123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:59:34.815560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131830+0000","last_change":"2024-02-26T06:25:18.224014+0000","last_active":"2024-02-26T06:26:42.131830+0000","last_peered":"2024-02-26T06:26:42.131830+0000","last_clean":"2024-02-26T06:26:42.131830+0000","last_became_active":"2024-02-26T06:25:18.223665+0000","last_became_peered":"2024-02-26T06:25:18.223665+0000","last_unstale":"2024-02-26T06:26:42.131830+0000","last_undegraded":"2024-02-26T06:26:42.131830+0000","last_fullsized":"2024-02-26T06:26:42.131830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:08.516188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851163+0000","last_change":"2024-02-26T06:25:17.209470+0000","last_active":"2024-02-26T06:26:13.851163+0000","last_peered":"2024-02-26T06:26:13.851163+0000","last_clean":"2024-02-26T06:26:13.851163+0000","last_became_active":"2024-02-26T06:25:17.209025+0000","last_became_peered":"2024-02-26T06:25:17.209025+0000","last_unstale":"2024-02-26T06:26:13.851163+0000","last_undegraded":"2024-02-26T06:26:13.851163+0000","last_fullsized":"2024-02-26T06:26:13.851163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:00.117824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112311+0000","last_change":"2024-02-26T06:25:18.210340+0000","last_active":"2024-02-26T06:26:42.112311+0000","last_peered":"2024-02-26T06:26:42.112311+0000","last_clean":"2024-02-26T06:26:42.112311+0000","last_became_active":"2024-02-26T06:25:18.209994+0000","last_became_peered":"2024-02-26T06:25:18.209994+0000","last_unstale":"2024-02-26T06:26:42.112311+0000","last_undegraded":"2024-02-26T06:26:42.112311+0000","last_fullsized":"2024-02-26T06:26:42.112311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:48.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'8","reported_seq":177,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:48.128168+0000","last_change":"2024-02-26T06:25:17.194406+0000","last_active":"2024-02-26T06:27:48.128168+0000","last_peered":"2024-02-26T06:27:48.128168+0000","last_clean":"2024-02-26T06:27:48.128168+0000","last_became_active":"2024-02-26T06:25:17.193971+0000","last_became_peered":"2024-02-26T06:25:17.193971+0000","last_unstale":"2024-02-26T06:27:48.128168+0000","last_undegraded":"2024-02-26T06:27:48.128168+0000","last_fullsized":"2024-02-26T06:27:48.128168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:36.724873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113095+0000","last_change":"2024-02-26T06:25:18.205758+0000","last_active":"2024-02-26T06:26:42.113095+0000","last_peered":"2024-02-26T06:26:42.113095+0000","last_clean":"2024-02-26T06:26:42.113095+0000","last_became_active":"2024-02-26T06:25:18.205415+0000","last_became_peered":"2024-02-26T06:25:18.205415+0000","last_unstale":"2024-02-26T06:26:42.113095+0000","last_undegraded":"2024-02-26T06:26:42.113095+0000","last_fullsized":"2024-02-26T06:26:42.113095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:05:14.969468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844759+0000","last_change":"2024-02-26T06:25:17.208123+0000","last_active":"2024-02-26T06:26:13.844759+0000","last_peered":"2024-02-26T06:26:13.844759+0000","last_clean":"2024-02-26T06:26:13.844759+0000","last_became_active":"2024-02-26T06:25:17.207698+0000","last_became_peered":"2024-02-26T06:25:17.207698+0000","last_unstale":"2024-02-26T06:26:13.844759+0000","last_undegraded":"2024-02-26T06:26:13.844759+0000","last_fullsized":"2024-02-26T06:26:13.844759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:33:26.400374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.113707+0000","last_change":"2024-02-26T06:26:13.862455+0000","last_active":"2024-02-26T06:26:42.113707+0000","last_peered":"2024-02-26T06:26:42.113707+0000","last_clean":"2024-02-26T06:26:42.113707+0000","last_became_active":"2024-02-26T06:26:13.861385+0000","last_became_peered":"2024-02-26T06:26:13.861385+0000","last_unstale":"2024-02-26T06:26:42.113707+0000","last_undegraded":"2024-02-26T06:26:42.113707+0000","last_fullsized":"2024-02-26T06:26:42.113707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:42:41.345104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.869092+0000","last_change":"2024-02-26T06:26:13.868540+0000","last_active":"2024-02-26T06:26:13.869092+0000","last_peered":"2024-02-26T06:26:13.869092+0000","last_clean":"2024-02-26T06:26:13.869092+0000","last_became_active":"2024-02-26T06:26:13.852149+0000","last_became_peered":"2024-02-26T06:26:13.852149+0000","last_unstale":"2024-02-26T06:26:13.869092+0000","last_undegraded":"2024-02-26T06:26:13.869092+0000","last_fullsized":"2024-02-26T06:26:13.869092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:14:30.745782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.101726+0000","last_change":"2024-02-26T06:25:18.202028+0000","last_active":"2024-02-26T06:26:42.101726+0000","last_peered":"2024-02-26T06:26:42.101726+0000","last_clean":"2024-02-26T06:26:42.101726+0000","last_became_active":"2024-02-26T06:25:18.201570+0000","last_became_peered":"2024-02-26T06:25:18.201570+0000","last_unstale":"2024-02-26T06:26:42.101726+0000","last_undegraded":"2024-02-26T06:26:42.101726+0000","last_fullsized":"2024-02-26T06:26:42.101726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:36.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.860103+0000","last_change":"2024-02-26T06:25:17.209687+0000","last_active":"2024-02-26T06:26:13.860103+0000","last_peered":"2024-02-26T06:26:13.860103+0000","last_clean":"2024-02-26T06:26:13.860103+0000","last_became_active":"2024-02-26T06:25:17.209282+0000","last_became_peered":"2024-02-26T06:25:17.209282+0000","last_unstale":"2024-02-26T06:26:13.860103+0000","last_undegraded":"2024-02-26T06:26:13.860103+0000","last_fullsized":"2024-02-26T06:26:13.860103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:40.345510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.105524+0000","last_change":"2024-02-26T06:26:13.872244+0000","last_active":"2024-02-26T06:26:42.105524+0000","last_peered":"2024-02-26T06:26:42.105524+0000","last_clean":"2024-02-26T06:26:42.105524+0000","last_became_active":"2024-02-26T06:26:13.866094+0000","last_became_peered":"2024-02-26T06:26:13.866094+0000","last_unstale":"2024-02-26T06:26:42.105524+0000","last_undegraded":"2024-02-26T06:26:42.105524+0000","last_fullsized":"2024-02-26T06:26:42.105524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:56:52.301479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":172,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.116789+0000","last_change":"2024-02-26T06:25:17.210601+0000","last_active":"2024-02-26T06:27:49.116789+0000","last_peered":"2024-02-26T06:27:49.116789+0000","last_clean":"2024-02-26T06:27:49.116789+0000","last_became_active":"2024-02-26T06:25:17.209917+0000","last_became_peered":"2024-02-26T06:25:17.209917+0000","last_unstale":"2024-02-26T06:27:49.116789+0000","last_undegraded":"2024-02-26T06:27:49.116789+0000","last_fullsized":"2024-02-26T06:27:49.116789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:52:47.127738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125601+0000","last_change":"2024-02-26T06:25:18.230451+0000","last_active":"2024-02-26T06:26:42.125601+0000","last_peered":"2024-02-26T06:26:42.125601+0000","last_clean":"2024-02-26T06:26:42.125601+0000","last_became_active":"2024-02-26T06:25:18.230091+0000","last_became_peered":"2024-02-26T06:25:18.230091+0000","last_unstale":"2024-02-26T06:26:42.125601+0000","last_undegraded":"2024-02-26T06:26:42.125601+0000","last_fullsized":"2024-02-26T06:26:42.125601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:03:30.414396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.094140+0000","last_change":"2024-02-26T06:25:18.204698+0000","last_active":"2024-02-26T06:26:42.094140+0000","last_peered":"2024-02-26T06:26:42.094140+0000","last_clean":"2024-02-26T06:26:42.094140+0000","last_became_active":"2024-02-26T06:25:18.204367+0000","last_became_peered":"2024-02-26T06:25:18.204367+0000","last_unstale":"2024-02-26T06:26:42.094140+0000","last_undegraded":"2024-02-26T06:26:42.094140+0000","last_fullsized":"2024-02-26T06:26:42.094140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:31.212713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'7","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.129482+0000","last_change":"2024-02-26T06:25:17.213045+0000","last_active":"2024-02-26T06:27:49.129482+0000","last_peered":"2024-02-26T06:27:49.129482+0000","last_clean":"2024-02-26T06:27:49.129482+0000","last_became_active":"2024-02-26T06:25:17.211446+0000","last_became_peered":"2024-02-26T06:25:17.211446+0000","last_unstale":"2024-02-26T06:27:49.129482+0000","last_undegraded":"2024-02-26T06:27:49.129482+0000","last_fullsized":"2024-02-26T06:27:49.129482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:05:30.006193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130696+0000","last_change":"2024-02-26T06:25:18.225997+0000","last_active":"2024-02-26T06:26:42.130696+0000","last_peered":"2024-02-26T06:26:42.130696+0000","last_clean":"2024-02-26T06:26:42.130696+0000","last_became_active":"2024-02-26T06:25:18.219655+0000","last_became_peered":"2024-02-26T06:25:18.219655+0000","last_unstale":"2024-02-26T06:26:42.130696+0000","last_undegraded":"2024-02-26T06:26:42.130696+0000","last_fullsized":"2024-02-26T06:26:42.130696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:14.463470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849961+0000","last_change":"2024-02-26T06:25:17.206586+0000","last_active":"2024-02-26T06:26:13.849961+0000","last_peered":"2024-02-26T06:26:13.849961+0000","last_clean":"2024-02-26T06:26:13.849961+0000","last_became_active":"2024-02-26T06:25:17.206121+0000","last_became_peered":"2024-02-26T06:25:17.206121+0000","last_unstale":"2024-02-26T06:26:13.849961+0000","last_undegraded":"2024-02-26T06:26:13.849961+0000","last_fullsized":"2024-02-26T06:26:13.849961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:09:35.043059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846775+0000","last_change":"2024-02-26T06:23:52.870855+0000","last_active":"2024-02-26T06:26:13.846775+0000","last_peered":"2024-02-26T06:26:13.846775+0000","last_clean":"2024-02-26T06:26:13.846775+0000","last_became_active":"2024-02-26T06:23:52.870285+0000","last_became_peered":"2024-02-26T06:23:52.870285+0000","last_unstale":"2024-02-26T06:26:13.846775+0000","last_undegraded":"2024-02-26T06:26:13.846775+0000","last_fullsized":"2024-02-26T06:26:13.846775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:20:02.657112+0000","last_clean_scrub_stamp":"2024-02-26T06:20:02.657112+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:43.748427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.121139+0000","last_change":"2024-02-26T06:25:18.207870+0000","last_active":"2024-02-26T06:26:42.121139+0000","last_peered":"2024-02-26T06:26:42.121139+0000","last_clean":"2024-02-26T06:26:42.121139+0000","last_became_active":"2024-02-26T06:25:18.206511+0000","last_became_peered":"2024-02-26T06:25:18.206511+0000","last_unstale":"2024-02-26T06:26:42.121139+0000","last_undegraded":"2024-02-26T06:26:42.121139+0000","last_fullsized":"2024-02-26T06:26:42.121139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:51.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.846885+0000","last_change":"2024-02-26T06:25:17.199239+0000","last_active":"2024-02-26T06:26:13.846885+0000","last_peered":"2024-02-26T06:26:13.846885+0000","last_clean":"2024-02-26T06:26:13.846885+0000","last_became_active":"2024-02-26T06:25:17.198849+0000","last_became_peered":"2024-02-26T06:25:17.198849+0000","last_unstale":"2024-02-26T06:26:13.846885+0000","last_undegraded":"2024-02-26T06:26:13.846885+0000","last_fullsized":"2024-02-26T06:26:13.846885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:36.627608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.845086+0000","last_change":"2024-02-26T06:25:17.213263+0000","last_active":"2024-02-26T06:26:13.845086+0000","last_peered":"2024-02-26T06:26:13.845086+0000","last_clean":"2024-02-26T06:26:13.845086+0000","last_became_active":"2024-02-26T06:25:17.212823+0000","last_became_peered":"2024-02-26T06:25:17.212823+0000","last_unstale":"2024-02-26T06:26:13.845086+0000","last_undegraded":"2024-02-26T06:26:13.845086+0000","last_fullsized":"2024-02-26T06:26:13.845086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:14.409830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.123071+0000","last_change":"2024-02-26T06:25:18.216617+0000","last_active":"2024-02-26T06:26:42.123071+0000","last_peered":"2024-02-26T06:26:42.123071+0000","last_clean":"2024-02-26T06:26:42.123071+0000","last_became_active":"2024-02-26T06:25:18.216147+0000","last_became_peered":"2024-02-26T06:25:18.216147+0000","last_unstale":"2024-02-26T06:26:42.123071+0000","last_undegraded":"2024-02-26T06:26:42.123071+0000","last_fullsized":"2024-02-26T06:26:42.123071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:21:07.471408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:45.258314+0000","last_change":"2024-02-26T06:25:17.219337+0000","last_active":"2024-02-26T06:27:45.258314+0000","last_peered":"2024-02-26T06:27:45.258314+0000","last_clean":"2024-02-26T06:27:45.258314+0000","last_became_active":"2024-02-26T06:25:17.218814+0000","last_became_peered":"2024-02-26T06:25:17.218814+0000","last_unstale":"2024-02-26T06:27:45.258314+0000","last_undegraded":"2024-02-26T06:27:45.258314+0000","last_fullsized":"2024-02-26T06:27:45.258314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:26.339888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126193+0000","last_change":"2024-02-26T06:25:18.211956+0000","last_active":"2024-02-26T06:26:42.126193+0000","last_peered":"2024-02-26T06:26:42.126193+0000","last_clean":"2024-02-26T06:26:42.126193+0000","last_became_active":"2024-02-26T06:25:18.211496+0000","last_became_peered":"2024-02-26T06:25:18.211496+0000","last_unstale":"2024-02-26T06:26:42.126193+0000","last_undegraded":"2024-02-26T06:26:42.126193+0000","last_fullsized":"2024-02-26T06:26:42.126193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:27.699510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.849969+0000","last_change":"2024-02-26T06:25:17.210703+0000","last_active":"2024-02-26T06:26:13.849969+0000","last_peered":"2024-02-26T06:26:13.849969+0000","last_clean":"2024-02-26T06:26:13.849969+0000","last_became_active":"2024-02-26T06:25:17.210299+0000","last_became_peered":"2024-02-26T06:25:17.210299+0000","last_unstale":"2024-02-26T06:26:13.849969+0000","last_undegraded":"2024-02-26T06:26:13.849969+0000","last_fullsized":"2024-02-26T06:26:13.849969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:39:50.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.128571+0000","last_change":"2024-02-26T06:25:18.232417+0000","last_active":"2024-02-26T06:26:42.128571+0000","last_peered":"2024-02-26T06:26:42.128571+0000","last_clean":"2024-02-26T06:26:42.128571+0000","last_became_active":"2024-02-26T06:25:18.231835+0000","last_became_peered":"2024-02-26T06:25:18.231835+0000","last_unstale":"2024-02-26T06:26:42.128571+0000","last_undegraded":"2024-02-26T06:26:42.128571+0000","last_fullsized":"2024-02-26T06:26:42.128571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:52.372290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.847127+0000","last_change":"2024-02-26T06:25:17.207987+0000","last_active":"2024-02-26T06:26:13.847127+0000","last_peered":"2024-02-26T06:26:13.847127+0000","last_clean":"2024-02-26T06:26:13.847127+0000","last_became_active":"2024-02-26T06:25:17.207708+0000","last_became_peered":"2024-02-26T06:25:17.207708+0000","last_unstale":"2024-02-26T06:26:13.847127+0000","last_undegraded":"2024-02-26T06:26:13.847127+0000","last_fullsized":"2024-02-26T06:26:13.847127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:53:44.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.098805+0000","last_change":"2024-02-26T06:25:18.200325+0000","last_active":"2024-02-26T06:26:42.098805+0000","last_peered":"2024-02-26T06:26:42.098805+0000","last_clean":"2024-02-26T06:26:42.098805+0000","last_became_active":"2024-02-26T06:25:18.199276+0000","last_became_peered":"2024-02-26T06:25:18.199276+0000","last_unstale":"2024-02-26T06:26:42.098805+0000","last_undegraded":"2024-02-26T06:26:42.098805+0000","last_fullsized":"2024-02-26T06:26:42.098805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:15:02.415549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.866330+0000","last_change":"2024-02-26T06:25:17.211190+0000","last_active":"2024-02-26T06:26:13.866330+0000","last_peered":"2024-02-26T06:26:13.866330+0000","last_clean":"2024-02-26T06:26:13.866330+0000","last_became_active":"2024-02-26T06:25:17.208935+0000","last_became_peered":"2024-02-26T06:25:17.208935+0000","last_unstale":"2024-02-26T06:26:13.866330+0000","last_undegraded":"2024-02-26T06:26:13.866330+0000","last_fullsized":"2024-02-26T06:26:13.866330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:34:35.215544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125635+0000","last_change":"2024-02-26T06:25:18.223735+0000","last_active":"2024-02-26T06:26:42.125635+0000","last_peered":"2024-02-26T06:26:42.125635+0000","last_clean":"2024-02-26T06:26:42.125635+0000","last_became_active":"2024-02-26T06:25:18.223380+0000","last_became_peered":"2024-02-26T06:25:18.223380+0000","last_unstale":"2024-02-26T06:26:42.125635+0000","last_undegraded":"2024-02-26T06:26:42.125635+0000","last_fullsized":"2024-02-26T06:26:42.125635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:18.484298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:30.989638+0000","last_change":"2024-02-26T06:25:17.203696+0000","last_active":"2024-02-26T06:27:30.989638+0000","last_peered":"2024-02-26T06:27:30.989638+0000","last_clean":"2024-02-26T06:27:30.989638+0000","last_became_active":"2024-02-26T06:25:17.203299+0000","last_became_peered":"2024-02-26T06:25:17.203299+0000","last_unstale":"2024-02-26T06:27:30.989638+0000","last_undegraded":"2024-02-26T06:27:30.989638+0000","last_fullsized":"2024-02-26T06:27:30.989638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:59.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129822+0000","last_change":"2024-02-26T06:25:18.229963+0000","last_active":"2024-02-26T06:26:42.129822+0000","last_peered":"2024-02-26T06:26:42.129822+0000","last_clean":"2024-02-26T06:26:42.129822+0000","last_became_active":"2024-02-26T06:25:18.229528+0000","last_became_peered":"2024-02-26T06:25:18.229528+0000","last_unstale":"2024-02-26T06:26:42.129822+0000","last_undegraded":"2024-02-26T06:26:42.129822+0000","last_fullsized":"2024-02-26T06:26:42.129822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:58:27.335021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.844858+0000","last_change":"2024-02-26T06:25:17.197421+0000","last_active":"2024-02-26T06:26:13.844858+0000","last_peered":"2024-02-26T06:26:13.844858+0000","last_clean":"2024-02-26T06:26:13.844858+0000","last_became_active":"2024-02-26T06:25:17.197064+0000","last_became_peered":"2024-02-26T06:25:17.197064+0000","last_unstale":"2024-02-26T06:26:13.844858+0000","last_undegraded":"2024-02-26T06:26:13.844858+0000","last_fullsized":"2024-02-26T06:26:13.844858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:39.024241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112189+0000","last_change":"2024-02-26T06:25:18.209163+0000","last_active":"2024-02-26T06:26:42.112189+0000","last_peered":"2024-02-26T06:26:42.112189+0000","last_clean":"2024-02-26T06:26:42.112189+0000","last_became_active":"2024-02-26T06:25:18.208804+0000","last_became_peered":"2024-02-26T06:25:18.208804+0000","last_unstale":"2024-02-26T06:26:42.112189+0000","last_undegraded":"2024-02-26T06:26:42.112189+0000","last_fullsized":"2024-02-26T06:26:42.112189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:50.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":177,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:27:49.005749+0000","last_change":"2024-02-26T06:25:17.202166+0000","last_active":"2024-02-26T06:27:49.005749+0000","last_peered":"2024-02-26T06:27:49.005749+0000","last_clean":"2024-02-26T06:27:49.005749+0000","last_became_active":"2024-02-26T06:25:17.201788+0000","last_became_peered":"2024-02-26T06:25:17.201788+0000","last_unstale":"2024-02-26T06:27:49.005749+0000","last_undegraded":"2024-02-26T06:27:49.005749+0000","last_fullsized":"2024-02-26T06:27:49.005749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:11.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1871,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.107689+0000","last_change":"2024-02-26T06:25:18.203987+0000","last_active":"2024-02-26T06:26:42.107689+0000","last_peered":"2024-02-26T06:26:42.107689+0000","last_clean":"2024-02-26T06:26:42.107689+0000","last_became_active":"2024-02-26T06:25:18.203128+0000","last_became_peered":"2024-02-26T06:25:18.203128+0000","last_unstale":"2024-02-26T06:26:42.107689+0000","last_undegraded":"2024-02-26T06:26:42.107689+0000","last_fullsized":"2024-02-26T06:26:42.107689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:32.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.851673+0000","last_change":"2024-02-26T06:25:17.209471+0000","last_active":"2024-02-26T06:26:13.851673+0000","last_peered":"2024-02-26T06:26:13.851673+0000","last_clean":"2024-02-26T06:26:13.851673+0000","last_became_active":"2024-02-26T06:25:17.209046+0000","last_became_peered":"2024-02-26T06:25:17.209046+0000","last_unstale":"2024-02-26T06:26:13.851673+0000","last_undegraded":"2024-02-26T06:26:13.851673+0000","last_fullsized":"2024-02-26T06:26:13.851673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:59.401697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.111993+0000","last_change":"2024-02-26T06:25:18.213289+0000","last_active":"2024-02-26T06:26:42.111993+0000","last_peered":"2024-02-26T06:26:42.111993+0000","last_clean":"2024-02-26T06:26:42.111993+0000","last_became_active":"2024-02-26T06:25:18.212908+0000","last_became_peered":"2024-02-26T06:25:18.212908+0000","last_unstale":"2024-02-26T06:26:42.111993+0000","last_undegraded":"2024-02-26T06:26:42.111993+0000","last_fullsized":"2024-02-26T06:26:42.111993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:03.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:13.879538+0000","last_change":"2024-02-26T06:26:13.878533+0000","last_active":"2024-02-26T06:26:13.879538+0000","last_peered":"2024-02-26T06:26:13.879538+0000","last_clean":"2024-02-26T06:26:13.879538+0000","last_became_active":"2024-02-26T06:26:13.866478+0000","last_became_peered":"2024-02-26T06:26:13.866478+0000","last_unstale":"2024-02-26T06:26:13.879538+0000","last_undegraded":"2024-02-26T06:26:13.879538+0000","last_fullsized":"2024-02-26T06:26:13.879538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:16.068405+0000","last_clean_scrub_stamp":"2024-02-26T06:25:16.068405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:00:23.896436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.099320+0000","last_change":"2024-02-26T06:25:18.205688+0000","last_active":"2024-02-26T06:26:42.099320+0000","last_peered":"2024-02-26T06:26:42.099320+0000","last_clean":"2024-02-26T06:26:42.099320+0000","last_became_active":"2024-02-26T06:25:18.205244+0000","last_became_peered":"2024-02-26T06:25:18.205244+0000","last_unstale":"2024-02-26T06:26:42.099320+0000","last_undegraded":"2024-02-26T06:26:42.099320+0000","last_fullsized":"2024-02-26T06:26:42.099320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:43:49.015564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100022+0000","last_change":"2024-02-26T06:25:18.199164+0000","last_active":"2024-02-26T06:26:42.100022+0000","last_peered":"2024-02-26T06:26:42.100022+0000","last_clean":"2024-02-26T06:26:42.100022+0000","last_became_active":"2024-02-26T06:25:18.198760+0000","last_became_peered":"2024-02-26T06:25:18.198760+0000","last_unstale":"2024-02-26T06:26:42.100022+0000","last_undegraded":"2024-02-26T06:26:42.100022+0000","last_fullsized":"2024-02-26T06:26:42.100022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:15.273586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.108252+0000","last_change":"2024-02-26T06:25:18.205157+0000","last_active":"2024-02-26T06:26:42.108252+0000","last_peered":"2024-02-26T06:26:42.108252+0000","last_clean":"2024-02-26T06:26:42.108252+0000","last_became_active":"2024-02-26T06:25:18.204811+0000","last_became_peered":"2024-02-26T06:25:18.204811+0000","last_unstale":"2024-02-26T06:26:42.108252+0000","last_undegraded":"2024-02-26T06:26:42.108252+0000","last_fullsized":"2024-02-26T06:26:42.108252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:24:27.360765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.124761+0000","last_change":"2024-02-26T06:25:18.220090+0000","last_active":"2024-02-26T06:26:42.124761+0000","last_peered":"2024-02-26T06:26:42.124761+0000","last_clean":"2024-02-26T06:26:42.124761+0000","last_became_active":"2024-02-26T06:25:18.219834+0000","last_became_peered":"2024-02-26T06:25:18.219834+0000","last_unstale":"2024-02-26T06:26:42.124761+0000","last_undegraded":"2024-02-26T06:26:42.124761+0000","last_fullsized":"2024-02-26T06:26:42.124761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:19:52.110711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.127390+0000","last_change":"2024-02-26T06:25:18.224679+0000","last_active":"2024-02-26T06:26:42.127390+0000","last_peered":"2024-02-26T06:26:42.127390+0000","last_clean":"2024-02-26T06:26:42.127390+0000","last_became_active":"2024-02-26T06:25:18.224227+0000","last_became_peered":"2024-02-26T06:25:18.224227+0000","last_unstale":"2024-02-26T06:26:42.127390+0000","last_undegraded":"2024-02-26T06:26:42.127390+0000","last_fullsized":"2024-02-26T06:26:42.127390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:55.909961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.126867+0000","last_change":"2024-02-26T06:25:18.221809+0000","last_active":"2024-02-26T06:26:42.126867+0000","last_peered":"2024-02-26T06:26:42.126867+0000","last_clean":"2024-02-26T06:26:42.126867+0000","last_became_active":"2024-02-26T06:25:18.221374+0000","last_became_peered":"2024-02-26T06:25:18.221374+0000","last_unstale":"2024-02-26T06:26:42.126867+0000","last_undegraded":"2024-02-26T06:26:42.126867+0000","last_fullsized":"2024-02-26T06:26:42.126867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:27:00.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.096844+0000","last_change":"2024-02-26T06:25:18.200807+0000","last_active":"2024-02-26T06:26:42.096844+0000","last_peered":"2024-02-26T06:26:42.096844+0000","last_clean":"2024-02-26T06:26:42.096844+0000","last_became_active":"2024-02-26T06:25:18.200478+0000","last_became_peered":"2024-02-26T06:25:18.200478+0000","last_unstale":"2024-02-26T06:26:42.096844+0000","last_undegraded":"2024-02-26T06:26:42.096844+0000","last_fullsized":"2024-02-26T06:26:42.096844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:27:54.032200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.106293+0000","last_change":"2024-02-26T06:25:18.202463+0000","last_active":"2024-02-26T06:26:42.106293+0000","last_peered":"2024-02-26T06:26:42.106293+0000","last_clean":"2024-02-26T06:26:42.106293+0000","last_became_active":"2024-02-26T06:25:18.202134+0000","last_became_peered":"2024-02-26T06:25:18.202134+0000","last_unstale":"2024-02-26T06:26:42.106293+0000","last_undegraded":"2024-02-26T06:26:42.106293+0000","last_fullsized":"2024-02-26T06:26:42.106293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:39.387775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.112460+0000","last_change":"2024-02-26T06:25:18.221545+0000","last_active":"2024-02-26T06:26:42.112460+0000","last_peered":"2024-02-26T06:26:42.112460+0000","last_clean":"2024-02-26T06:26:42.112460+0000","last_became_active":"2024-02-26T06:25:18.221034+0000","last_became_peered":"2024-02-26T06:25:18.221034+0000","last_unstale":"2024-02-26T06:26:42.112460+0000","last_undegraded":"2024-02-26T06:26:42.112460+0000","last_fullsized":"2024-02-26T06:26:42.112460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:55:10.035508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.130712+0000","last_change":"2024-02-26T06:25:18.216846+0000","last_active":"2024-02-26T06:26:42.130712+0000","last_peered":"2024-02-26T06:26:42.130712+0000","last_clean":"2024-02-26T06:26:42.130712+0000","last_became_active":"2024-02-26T06:25:18.216386+0000","last_became_peered":"2024-02-26T06:25:18.216386+0000","last_unstale":"2024-02-26T06:26:42.130712+0000","last_undegraded":"2024-02-26T06:26:42.130712+0000","last_fullsized":"2024-02-26T06:26:42.130712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:07.702357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.125683+0000","last_change":"2024-02-26T06:25:18.220180+0000","last_active":"2024-02-26T06:26:42.125683+0000","last_peered":"2024-02-26T06:26:42.125683+0000","last_clean":"2024-02-26T06:26:42.125683+0000","last_became_active":"2024-02-26T06:25:18.214906+0000","last_became_peered":"2024-02-26T06:25:18.214906+0000","last_unstale":"2024-02-26T06:26:42.125683+0000","last_undegraded":"2024-02-26T06:26:42.125683+0000","last_fullsized":"2024-02-26T06:26:42.125683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:04.471213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.131704+0000","last_change":"2024-02-26T06:25:18.221869+0000","last_active":"2024-02-26T06:26:42.131704+0000","last_peered":"2024-02-26T06:26:42.131704+0000","last_clean":"2024-02-26T06:26:42.131704+0000","last_became_active":"2024-02-26T06:25:18.221257+0000","last_became_peered":"2024-02-26T06:25:18.221257+0000","last_unstale":"2024-02-26T06:26:42.131704+0000","last_undegraded":"2024-02-26T06:26:42.131704+0000","last_fullsized":"2024-02-26T06:26:42.131704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:17.238524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.129590+0000","last_change":"2024-02-26T06:25:18.230381+0000","last_active":"2024-02-26T06:26:42.129590+0000","last_peered":"2024-02-26T06:26:42.129590+0000","last_clean":"2024-02-26T06:26:42.129590+0000","last_became_active":"2024-02-26T06:25:18.228916+0000","last_became_peered":"2024-02-26T06:25:18.228916+0000","last_unstale":"2024-02-26T06:26:42.129590+0000","last_undegraded":"2024-02-26T06:26:42.129590+0000","last_fullsized":"2024-02-26T06:26:42.129590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:18.739258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-26T06:26:42.100819+0000","last_change":"2024-02-26T06:25:18.203795+0000","last_active":"2024-02-26T06:26:42.100819+0000","last_peered":"2024-02-26T06:26:42.100819+0000","last_clean":"2024-02-26T06:26:42.100819+0000","last_became_active":"2024-02-26T06:25:18.203053+0000","last_became_peered":"2024-02-26T06:25:18.203053+0000","last_unstale":"2024-02-26T06:26:42.100819+0000","last_undegraded":"2024-02-26T06:26:42.100819+0000","last_fullsized":"2024-02-26T06:26:42.100819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T06:25:17.169415+0000","last_clean_scrub_stamp":"2024-02-26T06:25:17.169415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:41.959263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74392,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1496,"num_read_kb":829,"num_write":262,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383727,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":458239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94699999999999995,"5min":1.113,"15min":1.113},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.78000000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59699999999999998,"15min":0.56699999999999995},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.36499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66000000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:25:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.51900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.879},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80500000000000005}]},{"osd":5,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.751,"15min":0.751},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.71299999999999997,"5min":0.874,"15min":0.874},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.84999999999999998},"min":{"1min":0.56299999999999994,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.64600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":1.385,"15min":1.385},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.57999999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.92800000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.94099999999999995}]},{"osd":9,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.90100000000000002,"15min":0.90000000000000002},"min":{"1min":0.67400000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.179,"5min":1.238,"15min":1.238},"last":1.147},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.95099999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.151,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.04}]},{"osd":10,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.58299999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":1.044,"15min":1.044},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.73099999999999998},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.93500000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.91200000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1115232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":0.93700000000000006},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.4740000000000002,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45500000000000002,"15min":0.46300000000000002},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.373},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.627,"5min":1.093,"15min":1.093},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.752,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51600000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.167,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0920000000000001,"5min":1.458,"15min":1.458},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.52500000000000002,"15min":0.54300000000000004},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.56299999999999994,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.69499999999999995,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76900000000000002,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81100000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.081,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.73999999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82499999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60599999999999998}]},{"osd":9,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.004,"15min":1.034},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1279999999999999,"5min":1.256,"15min":1.256},"last":0.98699999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77100000000000002,"15min":0.78000000000000003},"min":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.629}]}]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.367,"15min":1.419},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.8979999999999999,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.3700000000000001,"15min":1.429},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7889999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.238}]},{"osd":1,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.1020000000000001,"15min":1.1759999999999999},"min":{"1min":0.70599999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.266,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.675},{"interface":"front","average":{"1min":1.117,"5min":1.2090000000000001,"15min":1.292},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.393,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1539999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4670000000000001,"15min":1.5429999999999999},"min":{"1min":0.69799999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8759999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3320000000000001,"15min":1.3919999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0209999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.468}]},{"osd":3,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.613,"15min":1.7350000000000001},"min":{"1min":0.749,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1400000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.736},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.7410000000000001,"15min":1.839},"min":{"1min":0.89800000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1869999999999998,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":2.238}]},{"osd":4,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7470000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1859999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":2.2669999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.766,"15min":1.9159999999999999},"min":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":2.665}]},{"osd":5,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.7709999999999999,"5min":2.012,"15min":2.0699999999999998},"min":{"1min":1.347,"5min":0.879,"15min":0.879},"max":{"1min":2.298,"5min":14.581,"15min":14.581},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.708,"5min":2.3500000000000001,"15min":2.7480000000000002},"min":{"1min":1.268,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3650000000000002,"5min":18.216000000000001,"15min":18.216000000000001},"last":2.3170000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.117,"15min":2.2599999999999998},"min":{"1min":1.2749999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.4199999999999999,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":3.4209999999999998},{"interface":"front","average":{"1min":1.8089999999999999,"5min":2.0339999999999998,"15min":2.194},"min":{"1min":1.167,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.5899999999999999,"5min":3.044,"15min":3.044},"last":3.5619999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.9339999999999999,"15min":2.0619999999999998},"min":{"1min":1.071,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.2269999999999999,"5min":3.036,"15min":3.036},"last":3.0870000000000002},{"interface":"front","average":{"1min":1.875,"5min":2.0510000000000002,"15min":2.1669999999999998},"min":{"1min":1.1499999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.4870000000000001,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":2.1230000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":2.0529999999999999,"5min":2.1589999999999998,"15min":2.2949999999999999},"min":{"1min":1.3859999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"max":{"1min":2.6160000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":2.1970000000000001},{"interface":"front","average":{"1min":1.889,"5min":2.0249999999999999,"15min":2.1579999999999999},"min":{"1min":1.359,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.492,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":2.355}]},{"osd":10,"last update":"Mon Feb 26 06:26:56 2024","interfaces":[{"interface":"back","average":{"1min":1.895,"5min":1.9390000000000001,"15min":1.9590000000000001},"min":{"1min":1.0389999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.4609999999999999,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":2.9630000000000001},{"interface":"front","average":{"1min":2.0939999999999999,"5min":2.1669999999999998,"15min":2.1949999999999998},"min":{"1min":1.089,"5min":1.089,"15min":1.089},"max":{"1min":2.6629999999999998,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":2.1619999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":456638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.44500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.505,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3069999999999999,"5min":1.478,"15min":1.478},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.47699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.752}]},{"osd":2,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.442},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45800000000000002,"15min":0.442},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.77400000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.55200000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1379999999999999,"5min":1.216,"15min":1.216},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1919999999999999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90000000000000002,"15min":0.92200000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.373,"15min":1.373},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.67700000000000005},"min":{"1min":0.56499999999999995,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64300000000000002}]},{"osd":9,"last update":"Mon Feb 26 06:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91500000000000004,"15min":0.91800000000000004},"min":{"1min":0.78100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.284,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.97499999999999998,"15min":1.0069999999999999},"min":{"1min":0.72899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.079}]},{"osd":10,"last update":"Mon Feb 26 06:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.73999999999999999},"min":{"1min":0.60499999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0329999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.55400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73399999999999999}]},{"osd":11,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.59599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.48799999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70699999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332164,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1050292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.139,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.40600000000000003},"min":{"1min":0.32000000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.52200000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.46300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.879,"15min":0.879},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61599999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.1240000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.619},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.497,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.78600000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.495},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67400000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40600000000000003}]},{"osd":5,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.76700000000000002,"5min":0.877,"15min":0.877},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Feb 26 06:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.90700000000000003,"15min":0.86899999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94999999999999996,"5min":6.9900000000000002,"15min":6.9900000000000002},"last":1.1910000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.753},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79700000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70799999999999996}]},{"osd":9,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":0.99199999999999999},"min":{"1min":0.73099999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0649999999999999,"15min":1.038},"min":{"1min":0.66800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1839999999999999,"5min":7.0380000000000003,"15min":7.0380000000000003},"last":1.0349999999999999}]},{"osd":10,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66500000000000004}]},{"osd":11,"last update":"Mon Feb 26 06:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.66100000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.90000000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69299999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.204,"15min":1.1970000000000001},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.923,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3080000000000001,"15min":1.3069999999999999},"min":{"1min":0.55900000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1749999999999998,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.734}]},{"osd":1,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0700000000000001,"15min":1.036},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6870000000000001,"5min":1.867,"15min":1.867},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.1519999999999999,"15min":1.117},"min":{"1min":0.69099999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.4750000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":1.2230000000000001}]},{"osd":2,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.23,"15min":1.1559999999999999},"min":{"1min":0.94199999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":2.6669999999999998},{"interface":"front","average":{"1min":1.331,"5min":1.3009999999999999,"15min":1.2370000000000001},"min":{"1min":0.65800000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.2530000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.1559999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.579,"15min":1.5369999999999999},"min":{"1min":0.82499999999999996,"5min":0.501,"15min":0.501},"max":{"1min":2.5830000000000002,"5min":3.113,"15min":3.113},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.625,"15min":1.647},"min":{"1min":0.71199999999999997,"5min":0.496,"15min":0.496},"max":{"1min":3.294,"5min":3.294,"15min":3.294},"last":2.802}]},{"osd":4,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.381,"15min":1.359},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":2.8319999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5740000000000001,"15min":1.514},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.492,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":2.1190000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5640000000000001,"15min":1.613},"min":{"1min":0.95699999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7210000000000001,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":2.4089999999999998},{"interface":"front","average":{"1min":1.657,"5min":1.619,"15min":1.6040000000000001},"min":{"1min":0.91600000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.786,"5min":2.798,"15min":2.798},"last":2.8919999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:27:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.7410000000000001,"15min":1.571},"min":{"1min":1.0049999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.6669999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":2.339},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.7030000000000001,"15min":1.625},"min":{"1min":0.72699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1760000000000002,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":2.5840000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.827,"15min":1.7410000000000001},"min":{"1min":1.1080000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":3.3620000000000001,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":2.4870000000000001},{"interface":"front","average":{"1min":1.867,"5min":1.7869999999999999,"15min":1.7509999999999999},"min":{"1min":1.0569999999999999,"5min":0.747,"15min":0.747},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":2.7410000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:27:28 2024","interfaces":[{"interface":"back","average":{"1min":2.004,"5min":2.04,"15min":2.234},"min":{"1min":0.94799999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":2.9700000000000002},{"interface":"front","average":{"1min":2.133,"5min":2.157,"15min":2.3380000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":3.1000000000000001,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":2.762}]},{"osd":10,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9339999999999999,"5min":2.024,"15min":2.113},"min":{"1min":1.1679999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":3.2330000000000001,"5min":3.2330000000000001,"15min":3.2330000000000001},"last":2.8530000000000002},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.9670000000000001,"15min":2.008},"min":{"1min":1.042,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":2.931}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":526360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68600000000000005,"15min":0.63200000000000001},"min":{"1min":0.443,"5min":0.436,"15min":0.374},"max":{"1min":1.0960000000000001,"5min":1.2,"15min":1.2},"last":0.443},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74199999999999999,"15min":0.66900000000000004},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1919999999999999,"5min":1.24,"15min":1.252},"last":0.41599999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95799999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84999999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.377}]},{"osd":3,"last update":"Mon Feb 26 06:27:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.441,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0880000000000001,"5min":1.214,"15min":1.214},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84599999999999997,"15min":0.80600000000000005},"min":{"1min":0.66100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Mon Feb 26 06:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.34599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.43099999999999999}]},{"osd":5,"last update":"Mon Feb 26 06:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.36499999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.43099999999999999}]},{"osd":6,"last update":"Mon Feb 26 06:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83399999999999996,"15min":0.86699999999999999},"min":{"1min":0.58499999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.077,"5min":1.196,"15min":1.196},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.90900000000000003},"min":{"1min":0.61399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.21,"15min":1.21},"last":0.60099999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.498,"15min":0.495},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67400000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39600000000000002}]},{"osd":8,"last update":"Mon Feb 26 06:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54700000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:27:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.84499999999999997},"min":{"1min":0.495,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.254,"5min":1.474,"15min":1.474},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.92900000000000005},"min":{"1min":0.69599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.628}]},{"osd":10,"last update":"Mon Feb 26 06:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.57499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":458621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.96599999999999997,"15min":0.81599999999999995},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.45900000000000002},"max":{"1min":1.379,"5min":1.569,"15min":1.569},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0069999999999999,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.40899999999999997},"max":{"1min":1.3320000000000001,"5min":3.516,"15min":3.516},"last":0.94099999999999995}]},{"osd":2,"last update":"Mon Feb 26 06:27:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0649999999999999,"15min":0.92200000000000004},"min":{"1min":0.64700000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2190000000000001,"15min":1.0109999999999999},"min":{"1min":0.89900000000000002,"5min":0.495,"15min":0.48599999999999999},"max":{"1min":1.9419999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3600000000000001}]},{"osd":3,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.6519999999999999,"15min":1.6679999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.3010000000000002,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.113},{"interface":"front","average":{"1min":1.782,"5min":1.794,"15min":1.796},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.3100000000000001,"5min":2.718,"15min":2.718},"last":1.486}]},{"osd":4,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.3460000000000001},"min":{"1min":0.78800000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.1160000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.139,"15min":1.133},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.621,"5min":2.069,"15min":2.069},"last":0.84799999999999998}]},{"osd":5,"last update":"Mon Feb 26 06:27:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3660000000000001,"15min":1.3220000000000001},"min":{"1min":0.77700000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.9059999999999999,"5min":2.653,"15min":2.653},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.7210000000000001,"15min":1.5600000000000001},"min":{"1min":0.97299999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.5710000000000002,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.431}]},{"osd":6,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.532,"15min":1.4530000000000001},"min":{"1min":0.745,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.46,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6890000000000001,"15min":1.575},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.383}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.6399999999999999,"15min":1.5880000000000001},"min":{"1min":1.1120000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.5189999999999999,"15min":1.4510000000000001},"min":{"1min":1.444,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.871}]},{"osd":8,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.7110000000000001,"15min":1.671},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.2749999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.294},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.875,"15min":1.8089999999999999},"min":{"1min":0.91200000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.552,"5min":2.911,"15min":2.911},"last":1.5229999999999999}]},{"osd":9,"last update":"Mon Feb 26 06:27:30 2024","interfaces":[{"interface":"back","average":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.173},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":3.254,"5min":3.254,"15min":3.254},"last":1.4610000000000001},{"interface":"front","average":{"1min":2.0950000000000002,"5min":2.1629999999999998,"15min":2.2690000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.7999999999999998,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.5620000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:27:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0259999999999998,"5min":2.0960000000000001,"15min":2.137},"min":{"1min":1.1200000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.8620000000000001,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.673,"15min":1.7270000000000001},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":2.2290000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.196}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.8600000000000001,"15min":1.8700000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4180000000000001,"5min":2.524,"15min":2.524},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6739999999999999,"15min":1.6819999999999999},"min":{"1min":0.89500000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.4500000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.403}]}]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":524761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72699999999999998,"15min":0.67100000000000004},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.163,"15min":1.163},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.68000000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1879999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Mon Feb 26 06:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80700000000000005,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75600000000000001,"15min":0.69899999999999995},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.505,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77100000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.39300000000000002},"min":{"1min":0.30299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85199999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.42899999999999999},"min":{"1min":0.35399999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88600000000000001,"15min":0.88500000000000001},"min":{"1min":0.74399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.90400000000000003,"15min":0.86499999999999999},"min":{"1min":0.64600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.319,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Mon Feb 26 06:24:31 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.51300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.749},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.89000000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80800000000000005}]},{"osd":9,"last update":"Mon Feb 26 06:27:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.86399999999999999,"15min":0.86199999999999999},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.04,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.93200000000000005,"15min":0.95799999999999996},"min":{"1min":0.67400000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1850000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.96099999999999997}]},{"osd":10,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.628},"min":{"1min":0.55600000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":1.089,"15min":1.089},"last":0.876}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0369999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73499999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.91600000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":456289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.1799999999999999,"15min":0.97899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.45,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.45},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.369,"15min":1.0549999999999999},"min":{"1min":0.71799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8440000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.6080000000000001}]},{"osd":1,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.1970000000000001,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.5199999999999996,"5min":7.5199999999999996,"15min":7.5199999999999996},"last":1.224},{"interface":"front","average":{"1min":1.143,"5min":1.2010000000000001,"15min":0.997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8300000000000001,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.3129999999999999}]},{"osd":2,"last update":"Mon Feb 26 06:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.252,"15min":1.077},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.748,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.262},{"interface":"front","average":{"1min":1.74,"5min":1.397,"15min":1.171},"min":{"1min":0.50800000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3760000000000003,"5min":7.3760000000000003,"15min":7.3760000000000003},"last":1.484}]},{"osd":4,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3380000000000001,"15min":1.1240000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.9450000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.028},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.4550000000000001,"15min":1.258},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0979999999999999,"5min":3.323,"15min":3.323},"last":1.411}]},{"osd":5,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.5960000000000001,"15min":1.391},"min":{"1min":0.84499999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.4039999999999999,"5min":7.4039999999999999,"15min":7.4039999999999999},"last":1.365},{"interface":"front","average":{"1min":1.51,"5min":1.5600000000000001,"15min":1.51},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4199999999999999,"5min":3.1349999999999998,"15min":5.7770000000000001},"last":1.54}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":1.829,"15min":1.7390000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2949999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.766},{"interface":"front","average":{"1min":1.556,"5min":1.671,"15min":1.4990000000000001},"min":{"1min":0.64600000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.2170000000000001,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.7490000000000001,"15min":1.7010000000000001},"min":{"1min":0.80000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2829999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.768,"5min":1.8999999999999999,"15min":1.8480000000000001},"min":{"1min":0.75,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3149999999999999,"5min":6.758,"15min":6.758},"last":1.5940000000000001}]},{"osd":8,"last update":"Mon Feb 26 06:24:53 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":1.7190000000000001,"15min":1.675},"min":{"1min":1.405,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.099},{"interface":"front","average":{"1min":2.165,"5min":1.9810000000000001,"15min":1.95},"min":{"1min":1.8029999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3520000000000001}]},{"osd":9,"last update":"Mon Feb 26 06:27:33 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.9870000000000001,"15min":2.145},"min":{"1min":0.72799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.6200000000000001,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.51},{"interface":"front","average":{"1min":1.847,"5min":2.0819999999999999,"15min":2.2349999999999999},"min":{"1min":0.95299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.5449999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":1.837}]},{"osd":10,"last update":"Mon Feb 26 06:26:58 2024","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":2.0750000000000002,"15min":2.1789999999999998},"min":{"1min":0.77400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.669,"5min":2.0049999999999999,"15min":2.153},"min":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.8140000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.6870000000000001}]},{"osd":11,"last update":"Mon Feb 26 06:27:20 2024","interfaces":[{"interface":"back","average":{"1min":1.929,"5min":1.8320000000000001,"15min":1.8160000000000001},"min":{"1min":1.1919999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.6549999999999998,"5min":3.085,"15min":3.085},"last":1.74},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7090000000000001,"15min":1.6910000000000001},"min":{"1min":0.79600000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.5059999999999998,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.7889999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.57699999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.23,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.71499999999999997,"15min":0.67000000000000004},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.41199999999999998,"15min":0.38200000000000001},"min":{"1min":0.32500000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.752,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.745},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.54300000000000004},"min":{"1min":0.46600000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":1.04,"15min":1.04},"last":0.5}]},{"osd":3,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78500000000000003,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.022,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":0.71499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1439999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.56899999999999995},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Feb 26 06:27:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82699999999999996,"15min":0.79100000000000004},"min":{"1min":0.48999999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.02,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.82999999999999996,"15min":0.84399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1659999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.48299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005}]},{"osd":8,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85199999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92200000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66600000000000004}]},{"osd":9,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86299999999999999,"15min":0.89500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.173,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94299999999999995,"15min":0.99299999999999999},"min":{"1min":0.69599999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.0580000000000001,"5min":1.665,"15min":1.665},"last":0.59899999999999998}]},{"osd":10,"last update":"Mon Feb 26 06:27:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68500000000000005,"15min":0.67400000000000004},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89500000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.47099999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78200000000000003}]},{"osd":11,"last update":"Mon Feb 26 06:27:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88400000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.81399999999999995,"15min":0.80900000000000005},"min":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.73299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1049047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.73899999999999999},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1339999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.754,"5min":0.76600000000000001,"15min":0.95699999999999996},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.03,"5min":1.99,"15min":7.2249999999999996},"last":0.73599999999999999}]},{"osd":1,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.21,"15min":1.21},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.5,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84499999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.54200000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.44},{"interface":"front","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.629,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.74399999999999999},"min":{"1min":0.54000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1830000000000001,"5min":1.298,"15min":1.298},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.82599999999999996,"15min":0.76600000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81499999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80500000000000005,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.46200000000000002}]},{"osd":6,"last update":"Mon Feb 26 06:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81899999999999995,"15min":0.85099999999999998},"min":{"1min":0.60499999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.075,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.65700000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.109,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.196,"15min":1.196},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.55500000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":1.177,"15min":1.177},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Feb 26 06:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.441,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.40600000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.502}]},{"osd":9,"last update":"Mon Feb 26 06:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93200000000000005,"15min":0.89100000000000001},"min":{"1min":0.73299999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.1910000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0129999999999999,"15min":0.996},"min":{"1min":0.79800000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.266,"5min":1.667,"15min":1.667},"last":0.76500000000000001}]},{"osd":10,"last update":"Mon Feb 26 06:26:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.53700000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.413,"15min":1.413},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T06:27:52.620 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:52.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 -- 172.21.15.6:0/3776748194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 msgr2=0x7fb590103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 --2- 172.21.15.6:0/3776748194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590103850 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb580009900 tx=0x7fb58002bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 -- 172.21.15.6:0/3776748194 shutdown_connections 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 --2- 172.21.15.6:0/3776748194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104fe0 0x7fb59010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 --2- 172.21.15.6:0/3776748194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104620 0x7fb590104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 --2- 172.21.15.6:0/3776748194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590103850 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.858+0000 7fb595800640 1 -- 172.21.15.6:0/3776748194 >> 172.21.15.6:0/3776748194 conn(0x7fb5900fe9e0 msgr2=0x7fb590100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.859+0000 7fb595800640 1 -- 172.21.15.6:0/3776748194 shutdown_connections 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.859+0000 7fb595800640 1 -- 172.21.15.6:0/3776748194 wait complete. 2024-02-26T06:27:52.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.859+0000 7fb595800640 1 Processor -- start 2024-02-26T06:27:52.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 -- start start 2024-02-26T06:27:52.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590078f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104620 0x7fb590079470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104fe0 0x7fb590075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590076230 con 0x7fb590103430 2024-02-26T06:27:52.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900763a0 con 0x7fb590104620 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb595800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590076560 con 0x7fb590104fe0 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb58ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590078f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb58ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590078f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41090/0 (socket says 172.21.15.6:41090) 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb58ea00640 1 -- 172.21.15.6:0/2001534826 learned_addr learned my addr 172.21.15.6:0/2001534826 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb58e000640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104620 0x7fb590079470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.860+0000 7fb58f400640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104fe0 0x7fb590075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58e000640 1 -- 172.21.15.6:0/2001534826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104fe0 msgr2=0x7fb590075aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58e000640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104fe0 0x7fb590075aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58e000640 1 -- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 msgr2=0x7fb590078f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58e000640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590078f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58e000640 1 -- 172.21.15.6:0/2001534826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb580009560 con 0x7fb590104620 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58ea00640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590078f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58f400640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104fe0 0x7fb590075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.861+0000 7fb58e000640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104620 0x7fb590079470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e9f0 tx=0x7fb57c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.862+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00ccc0 con 0x7fb590104620 2024-02-26T06:27:52.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.862+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb57c00ce20 con 0x7fb590104620 2024-02-26T06:27:52.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.862+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5901a59a0 con 0x7fb590104620 2024-02-26T06:27:52.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.862+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c004a00 con 0x7fb590104620 2024-02-26T06:27:52.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.862+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5901a5e60 con 0x7fb590104620 2024-02-26T06:27:52.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.863+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59010a7a0 con 0x7fb590104620 2024-02-26T06:27:52.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.866+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb57c0050b0 con 0x7fb590104620 2024-02-26T06:27:52.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.867+0000 7fb587400640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb5740a2150 0x7fb5740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:52.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.867+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb57c014070 con 0x7fb590104620 2024-02-26T06:27:52.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.867+0000 7fb58ea00640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb5740a2150 0x7fb5740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:52.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.868+0000 7fb58ea00640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb5740a2150 0x7fb5740a4610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb58002c850 tx=0x7fb58002cd20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:52.871+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb57c096d30 con 0x7fb590104620 2024-02-26T06:27:53.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.075+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb59010f020 con 0x7fb590104620 2024-02-26T06:27:53.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.078+0000 7fb587400640 1 -- 172.21.15.6:0/2001534826 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7fb57c09a3f0 con 0x7fb590104620 2024-02-26T06:27:53.078 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.080+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb5740a2150 msgr2=0x7fb5740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.080+0000 7fb595800640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb5740a2150 0x7fb5740a4610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb58002c850 tx=0x7fb58002cd20 comp rx=0 tx=0).stop 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.080+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104620 msgr2=0x7fb590079470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.080+0000 7fb595800640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104620 0x7fb590079470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e9f0 tx=0x7fb57c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.080+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 shutdown_connections 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb590104fe0 0x7fb590075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb5740a2150 0x7fb5740a4610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb590104620 0x7fb590079470 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 --2- 172.21.15.6:0/2001534826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb590103430 0x7fb590078f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 >> 172.21.15.6:0/2001534826 conn(0x7fb5900fe9e0 msgr2=0x7fb590107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 shutdown_connections 2024-02-26T06:27:53.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.081+0000 7fb595800640 1 -- 172.21.15.6:0/2001534826 wait complete. 2024-02-26T06:27:53.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:52 smithi006 ceph-mon[31790]: from='client.25355 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:53.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:52 smithi006 ceph-mon[31790]: from='client.25361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:53.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:53.101 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:52 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:53.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:27:53.105 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:27:53.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:52 smithi167 ceph-mon[36229]: from='client.25355 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:53.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:52 smithi167 ceph-mon[36229]: from='client.25361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:53.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:53.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:52 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 -- 172.21.15.6:0/1115535367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04141033b0 msgr2=0x7f04141037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 --2- 172.21.15.6:0/1115535367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04141033b0 0x7f04141037d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0404009900 tx=0x7f040402bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 -- 172.21.15.6:0/1115535367 shutdown_connections 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 --2- 172.21.15.6:0/1115535367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414104f60 0x7f041410b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 --2- 172.21.15.6:0/1115535367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04141045a0 0x7f0414104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 --2- 172.21.15.6:0/1115535367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04141033b0 0x7f04141037d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 -- 172.21.15.6:0/1115535367 >> 172.21.15.6:0/1115535367 conn(0x7f04140fe940 msgr2=0x7f0414100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 -- 172.21.15.6:0/1115535367 shutdown_connections 2024-02-26T06:27:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.309+0000 7f0419a00640 1 -- 172.21.15.6:0/1115535367 wait complete. 2024-02-26T06:27:53.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.310+0000 7f0419a00640 1 Processor -- start 2024-02-26T06:27:53.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.310+0000 7f0419a00640 1 -- start start 2024-02-26T06:27:53.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0419a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04141033b0 0x7f04141994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0419a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 0x7f04141999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0419a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0414104f60 0x7f04141a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0419a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141a0fa0 con 0x7f04141045a0 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0419a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141a1110 con 0x7f0414104f60 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0419a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141a12d0 con 0x7f04141033b0 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 0x7f04141999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 0x7f04141999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41108/0 (socket says 172.21.15.6:41108) 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 -- 172.21.15.6:0/145835194 learned_addr learned my addr 172.21.15.6:0/145835194 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04141033b0 0x7f04141994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0413400640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0414104f60 0x7f04141a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 -- 172.21.15.6:0/145835194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04141033b0 msgr2=0x7f04141994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04141033b0 0x7f04141994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 -- 172.21.15.6:0/145835194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0414104f60 msgr2=0x7f04141a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0414104f60 0x7f04141a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.311+0000 7f0412000640 1 -- 172.21.15.6:0/145835194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c009840 con 0x7f04141045a0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.312+0000 7f0412000640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 0x7f04141999f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f040000e9f0 tx=0x7f040000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.312+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400009720 con 0x7f04141045a0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.312+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0404009560 con 0x7f04141045a0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.312+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0400005500 con 0x7f04141045a0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.312+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400010430 con 0x7f04141045a0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.312+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04141a18e0 con 0x7f04141045a0 2024-02-26T06:27:53.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.314+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041410a7a0 con 0x7f04141045a0 2024-02-26T06:27:53.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.315+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f0400005000 con 0x7f04141045a0 2024-02-26T06:27:53.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.317+0000 7f040b400640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f03f80a2150 0x7f03f80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.317+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0400014070 con 0x7f04141045a0 2024-02-26T06:27:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.317+0000 7f0412a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f03f80a2150 0x7f03f80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.318+0000 7f0412a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f03f80a2150 0x7f03f80a4610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0404005ab0 tx=0x7f0404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.321+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04000970a0 con 0x7f04141045a0 2024-02-26T06:27:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:52 smithi090 ceph-mon[35681]: from='client.25355 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:52 smithi090 ceph-mon[35681]: from='client.25361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:53.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:52 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:53.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.543+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f041410ef70 con 0x7f04141045a0 2024-02-26T06:27:53.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.544+0000 7f040b400640 1 -- 172.21.15.6:0/145835194 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f040009a760 con 0x7f04141045a0 2024-02-26T06:27:53.544 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:27:53.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.546+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f03f80a2150 msgr2=0x7f03f80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.546+0000 7f0419a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f03f80a2150 0x7f03f80a4610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0404005ab0 tx=0x7f0404005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 msgr2=0x7f04141999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 0x7f04141999f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f040000e9f0 tx=0x7f040000edf0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 shutdown_connections 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f03f80a2150 0x7f03f80a4610 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0414104f60 0x7f04141a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04141045a0 0x7f04141999f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 --2- 172.21.15.6:0/145835194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04141033b0 0x7f04141994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 >> 172.21.15.6:0/145835194 conn(0x7f04140fe940 msgr2=0x7f0414107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:53.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 shutdown_connections 2024-02-26T06:27:53.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.547+0000 7f0419a00640 1 -- 172.21.15.6:0/145835194 wait complete. 2024-02-26T06:27:53.567 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:27:53.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 -- 172.21.15.6:0/2361963032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 msgr2=0x7fb2d8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 --2- 172.21.15.6:0/2361963032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d8103600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009900 tx=0x7fb2c002bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 -- 172.21.15.6:0/2361963032 shutdown_connections 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 --2- 172.21.15.6:0/2361963032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d8104d90 0x7fb2d810b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 --2- 172.21.15.6:0/2361963032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2d81043d0 0x7fb2d8104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 --2- 172.21.15.6:0/2361963032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d8103600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 -- 172.21.15.6:0/2361963032 >> 172.21.15.6:0/2361963032 conn(0x7fb2d80fe750 msgr2=0x7fb2d8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 -- 172.21.15.6:0/2361963032 shutdown_connections 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.772+0000 7fb2dca00640 1 -- 172.21.15.6:0/2361963032 wait complete. 2024-02-26T06:27:53.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.773+0000 7fb2dca00640 1 Processor -- start 2024-02-26T06:27:53.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 -- start start 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2d81043d0 0x7fb2d81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d8104d90 0x7fb2d81a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d81a0da0 con 0x7fb2d8104d90 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d81a0f10 con 0x7fb2d81031e0 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2dca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d81a10d0 con 0x7fb2d81043d0 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2d7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d8104d90 0x7fb2d81a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.774+0000 7fb2d7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d81992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55544/0 (socket says 172.21.15.6:55544) 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 -- 172.21.15.6:0/587305953 learned_addr learned my addr 172.21.15.6:0/587305953 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:53.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d6a00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2d81043d0 0x7fb2d81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 -- 172.21.15.6:0/587305953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2d81043d0 msgr2=0x7fb2d81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2d81043d0 0x7fb2d81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 -- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d8104d90 msgr2=0x7fb2d81a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d8104d90 0x7fb2d81a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 -- 172.21.15.6:0/587305953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0009560 con 0x7fb2d81031e0 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.775+0000 7fb2d7400640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d81992b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005ab0 tx=0x7fb2c0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.776+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0004aa0 con 0x7fb2d81031e0 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.776+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2c00052d0 con 0x7fb2d81031e0 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.776+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d81a1350 con 0x7fb2d81031e0 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.776+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0003e80 con 0x7fb2d81031e0 2024-02-26T06:27:53.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.776+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d81a18b0 con 0x7fb2d81031e0 2024-02-26T06:27:53.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.777+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d810a5d0 con 0x7fb2d81031e0 2024-02-26T06:27:53.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.779+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb2c0005440 con 0x7fb2d81031e0 2024-02-26T06:27:53.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.780+0000 7fb2cfe00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb2bc0a21d0 0x7fb2bc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:53.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.781+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb2c00f0160 con 0x7fb2d81031e0 2024-02-26T06:27:53.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.781+0000 7fb2d6a00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb2bc0a21d0 0x7fb2bc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:53.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.781+0000 7fb2d6a00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb2bc0a21d0 0x7fb2bc0a4690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2d819a7c0 tx=0x7fb2c8009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:53.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.784+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2c00b9070 con 0x7fb2d81031e0 2024-02-26T06:27:53.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.986+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2d802cdd0 con 0x7fb2d81031e0 2024-02-26T06:27:53.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.987+0000 7fb2cfe00640 1 -- 172.21.15.6:0/587305953 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2c00bc730 con 0x7fb2d81031e0 2024-02-26T06:27:53.988 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:27:53.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb2bc0a21d0 msgr2=0x7fb2bc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb2bc0a21d0 0x7fb2bc0a4690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2d819a7c0 tx=0x7fb2c8009680 comp rx=0 tx=0).stop 2024-02-26T06:27:53.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 msgr2=0x7fb2d81992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:53.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d81992b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005ab0 tx=0x7fb2c0004270 comp rx=0 tx=0).stop 2024-02-26T06:27:53.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 shutdown_connections 2024-02-26T06:27:53.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d8104d90 0x7fb2d81a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2d81043d0 0x7fb2d81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fb2bc0a21d0 0x7fb2bc0a4690 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 --2- 172.21.15.6:0/587305953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d81031e0 0x7fb2d81992b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:53.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 >> 172.21.15.6:0/587305953 conn(0x7fb2d80fe750 msgr2=0x7fb2d81077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:53.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 shutdown_connections 2024-02-26T06:27:53.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:53.990+0000 7fb2dca00640 1 -- 172.21.15.6:0/587305953 wait complete. 2024-02-26T06:27:54.010 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2001534826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:53 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/145835194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:54.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 -- 172.21.15.6:0/2164903211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 msgr2=0x7ff150103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 --2- 172.21.15.6:0/2164903211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff150103600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff140009900 tx=0x7ff14002bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:54.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 -- 172.21.15.6:0/2164903211 shutdown_connections 2024-02-26T06:27:54.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 --2- 172.21.15.6:0/2164903211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff150104d90 0x7ff15010b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 --2- 172.21.15.6:0/2164903211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1501043d0 0x7ff150104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 --2- 172.21.15.6:0/2164903211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff150103600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.237+0000 7ff155200640 1 -- 172.21.15.6:0/2164903211 >> 172.21.15.6:0/2164903211 conn(0x7ff1500fe750 msgr2=0x7ff150100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:54.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.238+0000 7ff155200640 1 -- 172.21.15.6:0/2164903211 shutdown_connections 2024-02-26T06:27:54.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.238+0000 7ff155200640 1 -- 172.21.15.6:0/2164903211 wait complete. 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 Processor -- start 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 -- start start 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff1501992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1501043d0 0x7ff1501997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150104d90 0x7ff1501a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1501a0da0 con 0x7ff1501031e0 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1501a0f10 con 0x7ff1501043d0 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff155200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1501a10d0 con 0x7ff150104d90 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff14f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff1501992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff14f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff1501992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41148/0 (socket says 172.21.15.6:41148) 2024-02-26T06:27:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff14f400640 1 -- 172.21.15.6:0/3629768112 learned_addr learned my addr 172.21.15.6:0/3629768112 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff14ea00640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1501043d0 0x7ff1501997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.239+0000 7ff14fe00640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150104d90 0x7ff1501a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff14f400640 1 -- 172.21.15.6:0/3629768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150104d90 msgr2=0x7ff1501a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff14f400640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150104d90 0x7ff1501a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff14f400640 1 -- 172.21.15.6:0/3629768112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1501043d0 msgr2=0x7ff1501997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff14f400640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1501043d0 0x7ff1501997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff14f400640 1 -- 172.21.15.6:0/3629768112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff148009840 con 0x7ff1501031e0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff14f400640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff1501992b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff140005ab0 tx=0x7ff140005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1400043d0 con 0x7ff1501031e0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff140009560 con 0x7ff1501031e0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.240+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1400052d0 con 0x7ff1501031e0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.241+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140003cd0 con 0x7ff1501031e0 2024-02-26T06:27:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.241+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1501a16d0 con 0x7ff1501031e0 2024-02-26T06:27:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.242+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15010a5a0 con 0x7ff1501031e0 2024-02-26T06:27:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.244+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7ff140005440 con 0x7ff1501031e0 2024-02-26T06:27:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-02-26T06:27:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2001534826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:27:54.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:27:54.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:54.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:53 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/145835194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.246+0000 7ff147e00640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1340a2150 0x7ff1340a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.246+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff1400efec0 con 0x7ff1501031e0 2024-02-26T06:27:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.246+0000 7ff14ea00640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1340a2150 0x7ff1340a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.247+0000 7ff14ea00640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1340a2150 0x7ff1340a4610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff15019a7c0 tx=0x7ff13c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.251+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1400b8dd0 con 0x7ff1501031e0 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2001534826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: from='mgr.14152 172.21.15.6:0/4045700127' entity='mgr.x' 2024-02-26T06:27:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:53 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/145835194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:27:54.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.451+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff15002cdd0 con 0x7ff1501031e0 2024-02-26T06:27:54.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.452+0000 7ff147e00640 1 -- 172.21.15.6:0/3629768112 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff1400bc490 con 0x7ff1501031e0 2024-02-26T06:27:54.452 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:27:54.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1340a2150 msgr2=0x7ff1340a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1340a2150 0x7ff1340a4610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff15019a7c0 tx=0x7ff13c005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 msgr2=0x7ff1501992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff1501992b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff140005ab0 tx=0x7ff140005e50 comp rx=0 tx=0).stop 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 shutdown_connections 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150104d90 0x7ff1501a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7ff1340a2150 0x7ff1340a4610 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1501043d0 0x7ff1501997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 --2- 172.21.15.6:0/3629768112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1501031e0 0x7ff1501992b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 >> 172.21.15.6:0/3629768112 conn(0x7ff1500fe750 msgr2=0x7ff150107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 shutdown_connections 2024-02-26T06:27:54.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.455+0000 7ff155200640 1 -- 172.21.15.6:0/3629768112 wait complete. 2024-02-26T06:27:54.474 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 -- 172.21.15.6:0/1393982236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec0104f70 msgr2=0x7f1ec010b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 --2- 172.21.15.6:0/1393982236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec0104f70 0x7f1ec010b800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009900 tx=0x7f1eb802bc40 comp rx=0 tx=0).stop 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 -- 172.21.15.6:0/1393982236 shutdown_connections 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 --2- 172.21.15.6:0/1393982236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec0104f70 0x7f1ec010b800 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 --2- 172.21.15.6:0/1393982236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec01045b0 0x7f1ec0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 --2- 172.21.15.6:0/1393982236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ec01033c0 0x7f1ec01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 -- 172.21.15.6:0/1393982236 >> 172.21.15.6:0/1393982236 conn(0x7f1ec00fe970 msgr2=0x7f1ec0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 -- 172.21.15.6:0/1393982236 shutdown_connections 2024-02-26T06:27:54.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.699+0000 7f1ec5400640 1 -- 172.21.15.6:0/1393982236 wait complete. 2024-02-26T06:27:54.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.700+0000 7f1ec5400640 1 Processor -- start 2024-02-26T06:27:54.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.700+0000 7f1ec5400640 1 -- start start 2024-02-26T06:27:54.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ec5400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 0x7f1ec0199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ec5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec01045b0 0x7f1ec0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ec5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ec0104f70 0x7f1ec01a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ec5400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01a0f00 con 0x7f1ec01033c0 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ec5400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01a1070 con 0x7f1ec0104f70 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ec5400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01a1230 con 0x7f1ec01045b0 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 0x7f1ec0199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 0x7f1ec0199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41176/0 (socket says 172.21.15.6:41176) 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 -- 172.21.15.6:0/2742458579 learned_addr learned my addr 172.21.15.6:0/2742458579 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebf200640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ec0104f70 0x7f1ec01a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1eb5e00640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec01045b0 0x7f1ec0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 -- 172.21.15.6:0/2742458579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec01045b0 msgr2=0x7f1ec0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec01045b0 0x7f1ec0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 -- 172.21.15.6:0/2742458579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ec0104f70 msgr2=0x7f1ec01a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.701+0000 7f1ebe800640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ec0104f70 0x7f1ec01a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1ebe800640 1 -- 172.21.15.6:0/2742458579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009840 con 0x7f1ec01033c0 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1ebe800640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 0x7f1ec0199410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009d20 tx=0x7f1eb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0009b40 con 0x7f1ec01033c0 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1eb0010b10 con 0x7f1ec01033c0 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea8009490 con 0x7f1ec01033c0 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0018cc0 con 0x7f1ec01033c0 2024-02-26T06:27:54.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.702+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb8009560 con 0x7f1ec01033c0 2024-02-26T06:27:54.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.703+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec010a6e0 con 0x7f1ec01033c0 2024-02-26T06:27:54.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.705+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f1eb0010c80 con 0x7f1ec01033c0 2024-02-26T06:27:54.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.707+0000 7f1eb7e00640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1ea40a21d0 0x7f1ea40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:54.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.707+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1eb0014070 con 0x7f1ec01033c0 2024-02-26T06:27:54.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.707+0000 7f1eb5e00640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1ea40a21d0 0x7f1ea40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:54.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.708+0000 7f1eb5e00640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1ea40a21d0 0x7f1ea40a4690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ec019a920 tx=0x7f1ea8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:54.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.711+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1eb00975a0 con 0x7f1ec01033c0 2024-02-26T06:27:54.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.953+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ec002ce40 con 0x7f1ec01033c0 2024-02-26T06:27:54.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.955+0000 7f1eb7e00640 1 -- 172.21.15.6:0/2742458579 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f1eb009ac60 con 0x7f1ec01033c0 2024-02-26T06:27:54.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:54.956 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:27:45.824981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:27:54.956 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1ea40a21d0 msgr2=0x7f1ea40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1ea40a21d0 0x7f1ea40a4690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ec019a920 tx=0x7f1ea8005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 msgr2=0x7f1ec0199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 0x7f1ec0199410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009d20 tx=0x7f1eb000c6a0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 shutdown_connections 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f1ea40a21d0 0x7f1ea40a4690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ec0104f70 0x7f1ec01a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec01045b0 0x7f1ec0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 --2- 172.21.15.6:0/2742458579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ec01033c0 0x7f1ec0199410 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:54.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 >> 172.21.15.6:0/2742458579 conn(0x7f1ec00fe970 msgr2=0x7f1ec01078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:54.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 shutdown_connections 2024-02-26T06:27:54.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:54.958+0000 7f1ec5400640 1 -- 172.21.15.6:0/2742458579 wait complete. 2024-02-26T06:27:54.980 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:27:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:54 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/587305953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:55.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:54 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3629768112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 -- 172.21.15.6:0/46896483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41032c0 msgr2=0x7f41f41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 --2- 172.21.15.6:0/46896483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41032c0 0x7f41f41036e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009900 tx=0x7f41e802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 -- 172.21.15.6:0/46896483 shutdown_connections 2024-02-26T06:27:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 --2- 172.21.15.6:0/46896483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 0x7f41f410b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 --2- 172.21.15.6:0/46896483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41044b0 0x7f41f4104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 --2- 172.21.15.6:0/46896483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41032c0 0x7f41f41036e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 -- 172.21.15.6:0/46896483 >> 172.21.15.6:0/46896483 conn(0x7f41f40fe850 msgr2=0x7f41f4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 -- 172.21.15.6:0/46896483 shutdown_connections 2024-02-26T06:27:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.197+0000 7f41fae00640 1 -- 172.21.15.6:0/46896483 wait complete. 2024-02-26T06:27:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 Processor -- start 2024-02-26T06:27:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 -- start start 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 0x7f41f4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41044b0 0x7f41f4199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 0x7f41f41a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41a0e40 con 0x7f41f41032c0 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41a0fb0 con 0x7f41f41044b0 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.198+0000 7f41fae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41a1170 con 0x7f41f4104e70 2024-02-26T06:27:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 0x7f41f4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 0x7f41f4199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41184/0 (socket says 172.21.15.6:41184) 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 -- 172.21.15.6:0/1220961251 learned_addr learned my addr 172.21.15.6:0/1220961251 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41fa400640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 0x7f41f41a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9000640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41044b0 0x7f41f4199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 -- 172.21.15.6:0/1220961251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 msgr2=0x7f41f41a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 0x7f41f41a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 -- 172.21.15.6:0/1220961251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41044b0 msgr2=0x7f41f4199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41044b0 0x7f41f4199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9a00640 1 -- 172.21.15.6:0/1220961251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0009490 con 0x7f41f41032c0 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41f9000640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41044b0 0x7f41f4199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.199+0000 7f41fa400640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 0x7f41f41a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.200+0000 7f41f9a00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 0x7f41f4199350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f41e8005ab0 tx=0x7f41e8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.200+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80043d0 con 0x7f41f41032c0 2024-02-26T06:27:55.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.200+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8009560 con 0x7f41f41032c0 2024-02-26T06:27:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.200+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41e80052d0 con 0x7f41f41032c0 2024-02-26T06:27:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.200+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8003d40 con 0x7f41f41032c0 2024-02-26T06:27:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.200+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f41a17c0 con 0x7f41f41032c0 2024-02-26T06:27:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.201+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f404ef50 con 0x7f41f41032c0 2024-02-26T06:27:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.203+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f41e8005440 con 0x7f41f41032c0 2024-02-26T06:27:55.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.205+0000 7f41f2000640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f41dc0a2140 0x7f41dc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.205+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f41e80efe70 con 0x7f41f41032c0 2024-02-26T06:27:55.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.205+0000 7f41f9000640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f41dc0a2140 0x7f41dc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.206+0000 7f41f9000640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f41dc0a2140 0x7f41dc0a4600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f41f419a860 tx=0x7f41e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.209+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41e80b8de0 con 0x7f41f41032c0 2024-02-26T06:27:55.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:54 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/587305953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:55.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:54 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3629768112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:54 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/587305953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:27:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:54 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3629768112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:27:55.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.409+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41f410a5e0 con 0x7f41f41032c0 2024-02-26T06:27:55.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.414+0000 7f41f2000640 1 -- 172.21.15.6:0/1220961251 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f41e80bc4a0 con 0x7f41f41032c0 2024-02-26T06:27:55.414 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:27:55.415 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:26:13.838432+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f41dc0a2140 msgr2=0x7f41dc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f41dc0a2140 0x7f41dc0a4600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f41f419a860 tx=0x7f41e0005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 msgr2=0x7f41f4199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 0x7f41f4199350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f41e8005ab0 tx=0x7f41e8005e50 comp rx=0 tx=0).stop 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 shutdown_connections 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f4104e70 0x7f41f41a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f41dc0a2140 0x7f41dc0a4600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f41044b0 0x7f41f4199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 --2- 172.21.15.6:0/1220961251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f41032c0 0x7f41f4199350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 >> 172.21.15.6:0/1220961251 conn(0x7f41f40fe850 msgr2=0x7f41f41077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 shutdown_connections 2024-02-26T06:27:55.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.417+0000 7f41fae00640 1 -- 172.21.15.6:0/1220961251 wait complete. 2024-02-26T06:27:55.437 DEBUG:tasks.fwd_scrub:Yielding 2024-02-26T06:27:55.437 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T06:27:55.447 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:27:55.447 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:27:55.447 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-26T06:27:55.448 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-26T06:27:55.448 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:27:55.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:27:55.448 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:27:55.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 -- 172.21.15.6:0/2797015432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 msgr2=0x7faba8106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 --2- 172.21.15.6:0/2797015432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba8106960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab94009860 tx=0x7fab9402bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 -- 172.21.15.6:0/2797015432 shutdown_connections 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 --2- 172.21.15.6:0/2797015432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faba8106ea0 0x7faba81092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 --2- 172.21.15.6:0/2797015432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba8106960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 --2- 172.21.15.6:0/2797015432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8101c40 0x7faba8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 -- 172.21.15.6:0/2797015432 >> 172.21.15.6:0/2797015432 conn(0x7faba80fb650 msgr2=0x7faba80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 -- 172.21.15.6:0/2797015432 shutdown_connections 2024-02-26T06:27:55.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.651+0000 7fabadc00640 1 -- 172.21.15.6:0/2797015432 wait complete. 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.652+0000 7fabadc00640 1 Processor -- start 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.652+0000 7fabadc00640 1 -- start start 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.652+0000 7fabadc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faba8101c40 0x7faba81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.652+0000 7fabadc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba81999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.652+0000 7fabadc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8106ea0 0x7faba81a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7fabadc00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81a0f90 con 0x7faba8104570 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7fabadc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81a1100 con 0x7faba8101c40 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7fabadc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81a12c0 con 0x7faba8106ea0 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba81999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faba8101c40 0x7faba81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba81999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41210/0 (socket says 172.21.15.6:41210) 2024-02-26T06:27:55.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 -- 172.21.15.6:0/840960605 learned_addr learned my addr 172.21.15.6:0/840960605 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba7400640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8106ea0 0x7faba81a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 -- 172.21.15.6:0/840960605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8106ea0 msgr2=0x7faba81a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8106ea0 0x7faba81a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 -- 172.21.15.6:0/840960605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faba8101c40 msgr2=0x7faba81994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faba8101c40 0x7faba81994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.653+0000 7faba6000640 1 -- 172.21.15.6:0/840960605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009490 con 0x7faba8104570 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.654+0000 7faba6000640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba81999e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab9402c0b0 tx=0x7fab94005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.654+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9402c510 con 0x7faba8104570 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.654+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab9402cb50 con 0x7faba8104570 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.654+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94003850 con 0x7faba8104570 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.654+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba0009900 con 0x7faba8104570 2024-02-26T06:27:55.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.654+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab94009490 con 0x7faba8104570 2024-02-26T06:27:55.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.656+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba81089f0 con 0x7faba8104570 2024-02-26T06:27:55.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.662+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fab9403b040 con 0x7faba8104570 2024-02-26T06:27:55.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.663+0000 7fab9f400640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fab8c0a21c0 0x7fab8c0a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.664+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fab940ef820 con 0x7faba8104570 2024-02-26T06:27:55.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.664+0000 7faba6a00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fab8c0a21c0 0x7fab8c0a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.665+0000 7faba6a00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fab8c0a21c0 0x7fab8c0a4680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab98009d60 tx=0x7fab98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.669+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab940b8840 con 0x7faba8104570 2024-02-26T06:27:55.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 -- 172.21.15.6:0/4215401207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 msgr2=0x7f2514073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 --2- 172.21.15.6:0/4215401207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f2514073610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2508009930 tx=0x7f250802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 -- 172.21.15.6:0/4215401207 shutdown_connections 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 --2- 172.21.15.6:0/4215401207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251410d760 0x7f251410fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 --2- 172.21.15.6:0/4215401207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f2514074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 --2- 172.21.15.6:0/4215401207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f2514073610 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 -- 172.21.15.6:0/4215401207 >> 172.21.15.6:0/4215401207 conn(0x7f25140fc700 msgr2=0x7f25140feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 -- 172.21.15.6:0/4215401207 shutdown_connections 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.670+0000 7f251b800640 1 -- 172.21.15.6:0/4215401207 wait complete. 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 Processor -- start 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 -- start start 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f251419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f251419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251410d760 0x7f25141a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141a5360 con 0x7f25140751c0 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141a54d0 con 0x7f2514073be0 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.671+0000 7f251b800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141a5690 con 0x7f251410d760 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f251419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f251419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55630/0 (socket says 172.21.15.6:55630) 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 -- 172.21.15.6:0/1900326345 learned_addr learned my addr 172.21.15.6:0/1900326345 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2513e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f251419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2519800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251410d760 0x7f25141a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 -- 172.21.15.6:0/1900326345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251410d760 msgr2=0x7f25141a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251410d760 0x7f25141a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 -- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 msgr2=0x7f251419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f251419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2518e00640 1 -- 172.21.15.6:0/1900326345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500009490 con 0x7f2514073be0 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.672+0000 7f2513e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f251419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.673+0000 7f2518e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f251419d870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2508000c00 tx=0x7f2508005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.673+0000 7f2511600640 1 -- 172.21.15.6:0/1900326345 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080043d0 con 0x7f2514073be0 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.673+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2508009560 con 0x7f2514073be0 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.673+0000 7f2511600640 1 -- 172.21.15.6:0/1900326345 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25080052d0 con 0x7f2514073be0 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.673+0000 7f2511600640 1 -- 172.21.15.6:0/1900326345 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508003da0 con 0x7f2514073be0 2024-02-26T06:27:55.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.673+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141a5ce0 con 0x7f2514073be0 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.677+0000 7f251e80f740 1 Processor -- start 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.677+0000 7f251e80f740 1 -- start start 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f251e80f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ecd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f251e80f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2511600640 1 -- 172.21.15.6:0/1900326345 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f2508005440 con 0x7f2514073be0 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f251e80f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ed4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f251e80f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21e892d0 con 0x563c21ecd920 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f251e80f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21e3be20 con 0x563c21ed2a90 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f251e80f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21e39350 con 0x563c21eccf30 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2518e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2519800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ecd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2519800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ecd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:44902/0 (socket says 172.21.15.6:44902) 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2518e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41238/0 (socket says 172.21.15.6:41238) 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2519800640 1 -- 172.21.15.6:0/708603031 learned_addr learned my addr 172.21.15.6:0/708603031 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.678+0000 7f2513e00640 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ed4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2519800640 1 -- 172.21.15.6:0/708603031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 msgr2=0x563c21ed4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2519800640 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ed4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2519800640 1 -- 172.21.15.6:0/708603031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 msgr2=0x563c21ed2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2519800640 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2519800640 1 -- 172.21.15.6:0/708603031 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c21dfdc50 con 0x563c21eccf30 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2518e00640 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.679+0000 7f2519800640 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ecd350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f250c009fd0 tx=0x7f250c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18693682e579759 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f2506000640 1 -- 172.21.15.6:0/708603031 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c004d20 con 0x563c21eccf30 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f2506000640 1 -- 172.21.15.6:0/708603031 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f250c004e80 con 0x563c21eccf30 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f2506000640 1 -- 172.21.15.6:0/708603031 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c005280 con 0x563c21eccf30 2024-02-26T06:27:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f2511600640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f24fc0a2150 0x7f24fc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f2511600640 1 -- 172.21.15.6:0/1900326345 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f25080f05d0 con 0x7f2514073be0 2024-02-26T06:27:55.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f2518e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f24fc0a2150 0x7f24fc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f251e80f740 1 -- 172.21.15.6:0/708603031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 msgr2=0x563c21ecd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.680+0000 7f251e80f740 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ecd350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f250c009fd0 tx=0x7f250c0118d0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 -- 172.21.15.6:0/708603031 shutdown_connections 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ed4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 --2- 172.21.15.6:0/708603031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ecd350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 -- 172.21.15.6:0/708603031 >> 172.21.15.6:0/708603031 conn(0x563c21ec84a0 msgr2=0x563c21ec88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 -- 172.21.15.6:0/708603031 shutdown_connections 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 -- 172.21.15.6:0/708603031 wait complete. 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f2518e00640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f24fc0a2150 0x7f24fc0a4610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x563c21ecdd20 tx=0x7f25080f60e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 -- start start 2024-02-26T06:27:55.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.681+0000 7f251e80f740 1 Processor -- start 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ede690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21e34b30 con 0x563c21ecd920 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21dd9fa0 con 0x563c21ed2a90 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21dd9c40 con 0x563c21eccf30 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f251e80f740 10 client.0 _ll_register_callbacks cb 0x563c21b45c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f2519800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ede690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f2513e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f2519800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ede690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55650/0 (socket says 172.21.15.6:55650) 2024-02-26T06:27:55.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f2519800640 1 -- 172.21.15.6:0/1651641510 learned_addr learned my addr 172.21.15.6:0/1651641510 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.682+0000 7f2519800640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2519800640 1 -- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 msgr2=0x563c21ede690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2519800640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ede690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2519800640 1 -- 172.21.15.6:0/1651641510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 msgr2=0x563c21ed7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2519800640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2519800640 1 -- 172.21.15.6:0/1651641510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c009c80 con 0x563c21eccf30 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2513e00640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f2519800640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed7110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f250c011dd0 tx=0x7f250c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2acc83528651dc8c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f24faa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f24faa00640 1 -- 172.21.15.6:0/1651641510 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00f4e0 con 0x563c21eccf30 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.683+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c21b85410 con 0x563c21eccf30 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f24faa00640 1 -- 172.21.15.6:0/1651641510 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f250c00f640 con 0x563c21eccf30 2024-02-26T06:27:55.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f24faa00640 1 -- 172.21.15.6:0/1651641510 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250c00f800 con 0x563c21eccf30 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f251e80f740 10 client.25402 fetch_fsmap learned FSMap version 27 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c21e331b0 con 0x563c21eccf30 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f24faa00640 1 -- 172.21.15.6:0/1651641510 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f250c037020 con 0x563c21eccf30 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f24faa00640 1 -- 172.21.15.6:0/1651641510 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f250c0289f0 con 0x563c21eccf30 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f251e80f740 10 client.25402 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.684+0000 7f251e80f740 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.685+0000 7f251e80f740 20 client.25402 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.685+0000 7f251e80f740 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21ee3340 0x563c21ee5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.685+0000 7f251e80f740 4 client.25402 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.685+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c21b717b0 con 0x563c21ee3340 2024-02-26T06:27:55.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.685+0000 7f2513e00640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21ee3340 0x563c21ee5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.686+0000 7f2513e00640 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21ee3340 0x563c21ee5760 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77452ca45fd8d7a3 server_cookie=7ad75b6eb60c0156 in_seq=0 out_seq=0 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.686+0000 7f24faa00640 10 client.25402 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.686+0000 7f24faa00640 1 -- 172.21.15.6:0/1651641510 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c21b717b0 con 0x563c21ee3340 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.686+0000 7f24faa00640 10 client.25402 handle_command_reply: tid=0 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.686+0000 7f251e80f740 1 client.25402 shutdown 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 msgr2=0x563c21ed7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed7110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f250c011dd0 tx=0x7f250c005fd0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 shutdown_connections 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21ee3340 0x563c21ee5760 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ed2a90 0x563c21ede690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ecd920 0x563c21ed7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 --2- 172.21.15.6:0/1651641510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed7110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f24faa00640 0 client.25402 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 >> 172.21.15.6:0/1651641510 conn(0x563c21ec84a0 msgr2=0x563c21ec76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 shutdown_connections 2024-02-26T06:27:55.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 1 -- 172.21.15.6:0/1651641510 wait complete. 2024-02-26T06:27:55.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.687+0000 7f251e80f740 20 client.25402 trim_cache size 0 max 16384 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.703+0000 7f251e80f740 1 Processor -- start 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 -- start start 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 0x563c21ede620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21ed7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21b7d0c0 con 0x563c21ed2a90 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21b717b0 con 0x563c21ecd920 2024-02-26T06:27:55.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f251e80f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21ecd7a0 con 0x563c21eccf30 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f2513e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 0x563c21ede620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f2519800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f2513e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 0x563c21ede620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55656/0 (socket says 172.21.15.6:55656) 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f2513e00640 1 -- 172.21.15.6:0/5546207 learned_addr learned my addr 172.21.15.6:0/5546207 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.704+0000 7f2519800640 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21ed7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2513e00640 1 -- 172.21.15.6:0/5546207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 msgr2=0x563c21ed6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2513e00640 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2513e00640 1 -- 172.21.15.6:0/5546207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 msgr2=0x563c21ed7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2513e00640 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21ed7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2513e00640 1 -- 172.21.15.6:0/5546207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c21dcfa60 con 0x563c21ecd920 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2519800640 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21ed7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2513e00640 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 0x563c21ede620 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x563c21ee6070 tx=0x7f2500005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0189d509fa1004d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2505600640 1 -- 172.21.15.6:0/5546207 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500017070 con 0x563c21ecd920 2024-02-26T06:27:55.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2505600640 1 -- 172.21.15.6:0/5546207 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2500018040 con 0x563c21ecd920 2024-02-26T06:27:55.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.705+0000 7f2505600640 1 -- 172.21.15.6:0/5546207 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000054e0 con 0x563c21ecd920 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 -- 172.21.15.6:0/5546207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 msgr2=0x563c21ede620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 0x563c21ede620 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x563c21ee6070 tx=0x7f2500005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 -- 172.21.15.6:0/5546207 shutdown_connections 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21ed7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21ecd920 0x563c21ede620 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 --2- 172.21.15.6:0/5546207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21eccf30 0x563c21ed6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 -- 172.21.15.6:0/5546207 >> 172.21.15.6:0/5546207 conn(0x563c21ec84a0 msgr2=0x563c21ec8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 -- 172.21.15.6:0/5546207 shutdown_connections 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.706+0000 7f251e80f740 1 -- 172.21.15.6:0/5546207 wait complete. 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 -- start start 2024-02-26T06:27:55.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 Processor -- start 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21eccf30 0x563c21edd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 0x563c21edd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21f39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21eddf00 con 0x563c21ed2a90 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21ede070 con 0x563c21eccf30 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21f39da0 con 0x563c21ecd920 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.707+0000 7f251e80f740 10 client.0 _ll_register_callbacks cb 0x7f250c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2519800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21eccf30 0x563c21edd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 0x563c21edd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2519800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21eccf30 0x563c21edd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55662/0 (socket says 172.21.15.6:55662) 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 0x563c21edd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:44918/0 (socket says 172.21.15.6:44918) 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2519800640 1 -- 172.21.15.6:0/3708161913 learned_addr learned my addr 172.21.15.6:0/3708161913 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2519800640 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21f39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 -- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21eccf30 msgr2=0x563c21edd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21eccf30 0x563c21edd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 -- 172.21.15.6:0/3708161913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 msgr2=0x563c21f39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21f39830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 -- 172.21.15.6:0/3708161913 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2500012450 con 0x563c21ecd920 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f2513e00640 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 0x563c21edd730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2500012720 tx=0x7f2500004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4665aebc64c0ed server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.708+0000 7f24faa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f24faa00640 1 -- 172.21.15.6:0/3708161913 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500017040 con 0x563c21ecd920 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c21b34860 con 0x563c21ecd920 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f24faa00640 1 -- 172.21.15.6:0/3708161913 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2500011970 con 0x563c21ecd920 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f24faa00640 1 -- 172.21.15.6:0/3708161913 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2500005730 con 0x563c21ecd920 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f251e80f740 10 client.25408 fetch_fsmap learned FSMap version 27 2024-02-26T06:27:55.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c21df5670 con 0x563c21ecd920 2024-02-26T06:27:55.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.709+0000 7f24faa00640 1 -- 172.21.15.6:0/3708161913 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563c21b34860 con 0x563c21ecd920 2024-02-26T06:27:55.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f24faa00640 1 -- 172.21.15.6:0/3708161913 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25000046c0 con 0x563c21ecd920 2024-02-26T06:27:55.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f251e80f740 10 client.25408 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:27:55.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f251e80f740 10 client.25408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f251e80f740 20 client.25408 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f251e80f740 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21f3b870 0x563c21f3dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f251e80f740 4 client.25408 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563c21ae0990 con 0x563c21f3b870 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.710+0000 7f2519800640 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21f3b870 0x563c21f3dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.711+0000 7f2519800640 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21f3b870 0x563c21f3dc90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5c2ab57082ad8a server_cookie=1cf10f90b8a61241 in_seq=0 out_seq=0 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.711+0000 7f24faa00640 10 client.25408 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f24faa00640 1 -- 172.21.15.6:0/3708161913 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563c21ae0990 con 0x563c21f3b870 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f24faa00640 10 client.25408 handle_command_reply: tid=0 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 client.25408 shutdown 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 msgr2=0x563c21edd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 0x563c21edd730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2500012720 tx=0x7f2500004320 comp rx=0 tx=0).stop 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 shutdown_connections 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x563c21f3b870 0x563c21f3dc90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c21ed2a90 0x563c21f39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c21ecd920 0x563c21edd730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f24faa00640 0 client.25408 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 --2- 172.21.15.6:0/3708161913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c21eccf30 0x563c21edd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.712+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 >> 172.21.15.6:0/3708161913 conn(0x563c21ec84a0 msgr2=0x563c21ed6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 shutdown_connections 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251e80f740 1 -- 172.21.15.6:0/3708161913 wait complete. 2024-02-26T06:27:55.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251e80f740 20 client.25408 trim_cache size 0 max 16384 2024-02-26T06:27:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:55.716 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-02-26T06:27:55.716 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "9fb93341-87ea-4d99-bff0-6d61ebf30093", 2024-02-26T06:27:55.716 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-02-26T06:27:55.716 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:55.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f24fc0a2150 msgr2=0x7f24fc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251b800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f24fc0a2150 0x7f24fc0a4610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x563c21ecdd20 tx=0x7f25080f60e0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 msgr2=0x7f251419d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.713+0000 7f251b800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f251419d870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2508000c00 tx=0x7f2508005e80 comp rx=0 tx=0).stop 2024-02-26T06:27:55.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 shutdown_connections 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251410d760 0x7f25141a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25140751c0 0x7f251419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f24fc0a2150 0x7f24fc0a4610 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 --2- 172.21.15.6:0/1900326345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2514073be0 0x7f251419d870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 >> 172.21.15.6:0/1900326345 conn(0x7f25140fc700 msgr2=0x7f2514100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 shutdown_connections 2024-02-26T06:27:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.714+0000 7f251b800640 1 -- 172.21.15.6:0/1900326345 wait complete. 2024-02-26T06:27:55.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:27:55.732 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:27:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.936+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7faba810ce20 con 0x7faba8104570 2024-02-26T06:27:55.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.952+0000 7feda1e00640 1 -- 172.21.15.6:0/3323760981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1087d0 msgr2=0x7fed9c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.952+0000 7feda1e00640 1 --2- 172.21.15.6:0/3323760981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1087d0 0x7fed9c108c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed88009860 tx=0x7fed8802bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 -- 172.21.15.6:0/3323760981 shutdown_connections 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 --2- 172.21.15.6:0/3323760981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c109190 0x7fed9c10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 --2- 172.21.15.6:0/3323760981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1087d0 0x7fed9c108c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 --2- 172.21.15.6:0/3323760981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 -- 172.21.15.6:0/3323760981 >> 172.21.15.6:0/3323760981 conn(0x7fed9c076570 msgr2=0x7fed9c0789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 -- 172.21.15.6:0/3323760981 shutdown_connections 2024-02-26T06:27:55.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.953+0000 7feda1e00640 1 -- 172.21.15.6:0/3323760981 wait complete. 2024-02-26T06:27:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 Processor -- start 2024-02-26T06:27:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 -- start start 2024-02-26T06:27:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c1087d0 0x7fed9c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c109190 0x7fed9c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1a5130 con 0x7fed9c1075e0 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1a52a0 con 0x7fed9c1087d0 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda1e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1a5460 con 0x7fed9c109190 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41296/0 (socket says 172.21.15.6:41296) 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.954+0000 7feda0a00640 1 -- 172.21.15.6:0/1987202822 learned_addr learned my addr 172.21.15.6:0/1987202822 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda1400640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c109190 0x7fed9c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7fed9be00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c1087d0 0x7fed9c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda0a00640 1 -- 172.21.15.6:0/1987202822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c109190 msgr2=0x7fed9c1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda0a00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c109190 0x7fed9c1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda0a00640 1 -- 172.21.15.6:0/1987202822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c1087d0 msgr2=0x7fed9c19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda0a00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c1087d0 0x7fed9c19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda0a00640 1 -- 172.21.15.6:0/1987202822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90009560 con 0x7fed9c1075e0 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7feda0a00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c19d730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed9000e9a0 tx=0x7fed9000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.955+0000 7fed99600640 1 -- 172.21.15.6:0/1987202822 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000cd80 con 0x7fed9c1075e0 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.956+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed88009490 con 0x7fed9c1075e0 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.956+0000 7fed99600640 1 -- 172.21.15.6:0/1987202822 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed90010910 con 0x7fed9c1075e0 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.956+0000 7fed99600640 1 -- 172.21.15.6:0/1987202822 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90018980 con 0x7fed9c1075e0 2024-02-26T06:27:55.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.956+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c1a5a90 con 0x7fed9c1075e0 2024-02-26T06:27:55.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 Processor -- start 2024-02-26T06:27:55.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 -- start start 2024-02-26T06:27:55.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7fed99600640 1 -- 172.21.15.6:0/1987202822 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fed90010a80 con 0x7fed9c1075e0 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c9647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c9698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c96c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c8d4120 con 0x55b98c9643c0 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c8d1650 con 0x55b98c964db0 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda63d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c8cce30 con 0x55b98c969f20 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.959+0000 7feda1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c9647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c96c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7feda1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c9647e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41302/0 (socket says 172.21.15.6:41302) 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c96c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:44952/0 (socket says 172.21.15.6:44952) 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7feda1400640 1 -- 172.21.15.6:0/3161506231 learned_addr learned my addr 172.21.15.6:0/3161506231 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7feda0a00640 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c9698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 -- 172.21.15.6:0/3161506231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 msgr2=0x55b98c9698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c9698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 -- 172.21.15.6:0/3161506231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 msgr2=0x55b98c9647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c9647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 -- 172.21.15.6:0/3161506231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b98c886030 con 0x55b98c969f20 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7feda1400640 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c9647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.960+0000 7fed9be00640 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c96c3b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed8802c0b0 tx=0x7fed88032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4d6b46b648d4d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.961+0000 7fed8dc00640 1 -- 172.21.15.6:0/3161506231 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88004300 con 0x55b98c969f20 2024-02-26T06:27:55.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.961+0000 7fed8dc00640 1 -- 172.21.15.6:0/3161506231 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed88034920 con 0x55b98c969f20 2024-02-26T06:27:55.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.961+0000 7fed8dc00640 1 -- 172.21.15.6:0/3161506231 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88003850 con 0x55b98c969f20 2024-02-26T06:27:55.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.961+0000 7feda63d6740 1 -- 172.21.15.6:0/3161506231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 msgr2=0x55b98c96c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.961+0000 7feda63d6740 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c96c3b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed8802c0b0 tx=0x7fed88032340 comp rx=0 tx=0).stop 2024-02-26T06:27:55.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7fed99600640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fed840a2150 0x7fed840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 -- 172.21.15.6:0/3161506231 shutdown_connections 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c96c3b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c9698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 --2- 172.21.15.6:0/3161506231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c9647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 -- 172.21.15.6:0/3161506231 >> 172.21.15.6:0/3161506231 conn(0x55b98c92c570 msgr2=0x55b98c95fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7fed99600640 1 -- 172.21.15.6:0/1987202822 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fed90014070 con 0x7fed9c1075e0 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 -- 172.21.15.6:0/3161506231 shutdown_connections 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 -- 172.21.15.6:0/3161506231 wait complete. 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda1400640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fed840a2150 0x7fed840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.962+0000 7feda63d6740 1 -- start start 2024-02-26T06:27:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 Processor -- start 2024-02-26T06:27:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda1400640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fed840a2150 0x7fed840a4610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed94005fd0 tx=0x7fed94009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c96e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c964db0 0x55b98c96e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c9759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c871e90 con 0x55b98c9643c0 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c871b30 con 0x55b98c969f20 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c63b1c0 con 0x55b98c964db0 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda63d6740 10 client.0 _ll_register_callbacks cb 0x55b98c885930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c96e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c96e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41310/0 (socket says 172.21.15.6:41310) 2024-02-26T06:27:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7feda0a00640 1 -- 172.21.15.6:0/491980521 learned_addr learned my addr 172.21.15.6:0/491980521 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.963+0000 7fed9be00640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c964db0 0x55b98c96e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda1400640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c9759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda0a00640 1 -- 172.21.15.6:0/491980521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c964db0 msgr2=0x55b98c96e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda0a00640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c964db0 0x55b98c96e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda0a00640 1 -- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 msgr2=0x55b98c9759c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda0a00640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c9759c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda0a00640 1 -- 172.21.15.6:0/491980521 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c110320 con 0x55b98c9643c0 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7feda0a00640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c96e440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c6a0 tx=0x7fed900972c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad557882d76e69b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7fed82000640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:27:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7fed82000640 1 -- 172.21.15.6:0/491980521 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9009a5b0 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.964+0000 7fed82000640 1 -- 172.21.15.6:0/491980521 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fed9009b660 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b98c609720 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7fed82000640 1 -- 172.21.15.6:0/491980521 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9009a710 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b98c8cb4b0 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7fed82000640 1 -- 172.21.15.6:0/491980521 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b98c609720 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7fed82000640 1 -- 172.21.15.6:0/491980521 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fed90018ae0 con 0x55b98c9643c0 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 10 client.15309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 20 client.15309 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c97a2f0 0x55b98c97c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 4 client.15309 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.965+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b98c658890 con 0x55b98c97a2f0 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.966+0000 7fed9be00640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c97a2f0 0x55b98c97c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.966+0000 7fed9be00640 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c97a2f0 0x55b98c97c700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218a7a830eb91892 server_cookie=74cfb86b0f6bcffe in_seq=0 out_seq=0 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.966+0000 7fed82000640 10 client.15309 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7fed82000640 1 -- 172.21.15.6:0/491980521 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b98c658890 con 0x55b98c97a2f0 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7fed82000640 10 client.15309 handle_command_reply: tid=0 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 client.15309 shutdown 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 msgr2=0x55b98c96e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c96e440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c6a0 tx=0x7fed900972c0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 shutdown_connections 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c97a2f0 0x55b98c97c700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c9759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c964db0 0x55b98c96e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 --2- 172.21.15.6:0/491980521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c96e440 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7fed82000640 0 client.15309 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.967+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 >> 172.21.15.6:0/491980521 conn(0x55b98c92c570 msgr2=0x55b98c95ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.968+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 shutdown_connections 2024-02-26T06:27:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.968+0000 7feda63d6740 1 -- 172.21.15.6:0/491980521 wait complete. 2024-02-26T06:27:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.968+0000 7feda63d6740 20 client.15309 trim_cache size 0 max 16384 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 Processor -- start 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 -- start start 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c9643c0 0x55b98c96e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 0x55b98c975950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c96e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c609720 con 0x55b98c964db0 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c960330 con 0x55b98c969f20 2024-02-26T06:27:55.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda63d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c95a300 con 0x55b98c9643c0 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7fed9be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 0x55b98c975950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7fed9be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 0x55b98c975950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41318/0 (socket says 172.21.15.6:41318) 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7fed9be00640 1 -- 172.21.15.6:0/2756160860 learned_addr learned my addr 172.21.15.6:0/2756160860 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.985+0000 7feda1400640 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c96e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7feda0a00640 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c9643c0 0x55b98c96e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed9be00640 1 -- 172.21.15.6:0/2756160860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c9643c0 msgr2=0x55b98c96e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed9be00640 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c9643c0 0x55b98c96e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed9be00640 1 -- 172.21.15.6:0/2756160860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 msgr2=0x55b98c96e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed9be00640 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c96e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed9be00640 1 -- 172.21.15.6:0/2756160860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b98c867920 con 0x55b98c964db0 2024-02-26T06:27:55.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7feda0a00640 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c9643c0 0x55b98c96e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed9be00640 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 0x55b98c975950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fed8802c180 tx=0x7fed8804a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f435b95a183cfa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.986+0000 7fed8d200640 1 -- 172.21.15.6:0/2756160860 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8802c8f0 con 0x55b98c964db0 2024-02-26T06:27:55.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7fed8d200640 1 -- 172.21.15.6:0/2756160860 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed88004040 con 0x55b98c964db0 2024-02-26T06:27:55.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7fed8d200640 1 -- 172.21.15.6:0/2756160860 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed880564c0 con 0x55b98c964db0 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 -- 172.21.15.6:0/2756160860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 msgr2=0x55b98c975950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 0x55b98c975950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fed8802c180 tx=0x7fed8804a040 comp rx=0 tx=0).stop 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 -- 172.21.15.6:0/2756160860 shutdown_connections 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c969f20 0x55b98c96e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c964db0 0x55b98c975950 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 --2- 172.21.15.6:0/2756160860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c9643c0 0x55b98c96e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 -- 172.21.15.6:0/2756160860 >> 172.21.15.6:0/2756160860 conn(0x55b98c92c570 msgr2=0x55b98c966310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 -- 172.21.15.6:0/2756160860 shutdown_connections 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.987+0000 7feda63d6740 1 -- 172.21.15.6:0/2756160860 wait complete. 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 -- start start 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 Processor -- start 2024-02-26T06:27:55.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c973a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c973f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c974ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c975400 con 0x55b98c9643c0 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c975570 con 0x55b98c964db0 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98c975730 con 0x55b98c969f20 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.988+0000 7feda63d6740 10 client.0 _ll_register_callbacks cb 0x7fed900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c973f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7feda0a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c973a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c973f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55722/0 (socket says 172.21.15.6:55722) 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 -- 172.21.15.6:0/2112287681 learned_addr learned my addr 172.21.15.6:0/2112287681 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7feda1400640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c974ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 -- 172.21.15.6:0/2112287681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 msgr2=0x55b98c974ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c974ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 -- 172.21.15.6:0/2112287681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 msgr2=0x55b98c973a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c973a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 -- 172.21.15.6:0/2112287681 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b98c8cb4b0 con 0x55b98c964db0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7feda0a00640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c973a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed9be00640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c973f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed88009780 tx=0x7fed88005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e7eeccf0405184 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.989+0000 7fed82000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7fed82000640 1 -- 172.21.15.6:0/2112287681 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88004480 con 0x55b98c964db0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7fed82000640 1 -- 172.21.15.6:0/2112287681 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fed8804d040 con 0x55b98c964db0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b98c95b550 con 0x55b98c964db0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7fed82000640 1 -- 172.21.15.6:0/2112287681 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed8802c6f0 con 0x55b98c964db0 2024-02-26T06:27:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 10 client.25445 fetch_fsmap learned FSMap version 27 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b98c886030 con 0x55b98c964db0 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7fed82000640 1 -- 172.21.15.6:0/2112287681 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b98c95b550 con 0x55b98c964db0 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7fed82000640 1 -- 172.21.15.6:0/2112287681 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fed88056240 con 0x55b98c964db0 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 10 client.25445 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 10 client.25445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 20 client.25445 populate_metadata read hostname 'smithi006' 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.990+0000 7feda63d6740 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c9d1700 0x55b98c9d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7feda63d6740 4 client.25445 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b98c5cc7e0 con 0x55b98c9d1700 2024-02-26T06:27:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7feda0a00640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c9d1700 0x55b98c9d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7feda0a00640 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c9d1700 0x55b98c9d3b20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163c50a74ab55f7 server_cookie=9286cfd657553959 in_seq=0 out_seq=0 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7fed82000640 10 client.25445 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7fed82000640 1 -- 172.21.15.6:0/2112287681 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b98c5cc7e0 con 0x55b98c9d1700 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.991+0000 7fed82000640 10 client.25445 handle_command_reply: tid=0 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 client.25445 shutdown 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 msgr2=0x55b98c973f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c973f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed88009780 tx=0x7fed88005780 comp rx=0 tx=0).stop 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 shutdown_connections 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55b98c9d1700 0x55b98c9d3b20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98c969f20 0x55b98c974ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7fed82000640 0 client.25445 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:27:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b98c964db0 0x55b98c973f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 --2- 172.21.15.6:0/2112287681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b98c9643c0 0x55b98c973a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 >> 172.21.15.6:0/2112287681 conn(0x55b98c92c570 msgr2=0x55b98c96e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 shutdown_connections 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 1 -- 172.21.15.6:0/2112287681 wait complete. 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.992+0000 7feda63d6740 20 client.25445 trim_cache size 0 max 16384 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stdout: "9fb93341-87ea-4d99-bff0-6d61ebf30093": { 2024-02-26T06:27:55.997 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-02-26T06:27:55.998 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9fb93341-87ea-4d99-bff0-6d61ebf30093", 2024-02-26T06:27:55.998 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-02-26T06:27:55.998 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:27:55.998 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:27:55.998 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fed840a2150 msgr2=0x7fed840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fed840a2150 0x7fed840a4610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed94005fd0 tx=0x7fed94009500 comp rx=0 tx=0).stop 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 msgr2=0x7fed9c19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c19d730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed9000e9a0 tx=0x7fed9000ee60 comp rx=0 tx=0).stop 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 shutdown_connections 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c109190 0x7fed9c1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fed840a2150 0x7fed840a4610 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed9c1087d0 0x7fed9c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 --2- 172.21.15.6:0/1987202822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed9c1075e0 0x7fed9c19d730 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 >> 172.21.15.6:0/1987202822 conn(0x7fed9c076570 msgr2=0x7fed9c10baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 shutdown_connections 2024-02-26T06:27:55.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:55.993+0000 7feda1e00640 1 -- 172.21.15.6:0/1987202822 wait complete. 2024-02-26T06:27:56.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:55 smithi006 ceph-mon[31790]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-02-26T06:27:56.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:55 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2742458579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:56.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:55 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1220961251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:56.013 INFO:tasks.cephfs.filesystem:scrub status for tag:9fb93341-87ea-4d99-bff0-6d61ebf30093 - {'path': '/', 'tag': '9fb93341-87ea-4d99-bff0-6d61ebf30093', 'options': 'recursive,force'} 2024-02-26T06:27:56.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:55 smithi167 ceph-mon[36229]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-02-26T06:27:56.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:55 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2742458579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:56.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:55 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1220961251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:55 smithi090 ceph-mon[35681]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-02-26T06:27:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:55 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2742458579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:27:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:55 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1220961251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:27:56.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.906+0000 7fab9f400640 1 -- 172.21.15.6:0/840960605 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7fab940bbf00 con 0x7faba8104570 2024-02-26T06:27:56.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fab8c0a21c0 msgr2=0x7fab8c0a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:56.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fab8c0a21c0 0x7fab8c0a4680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab98009d60 tx=0x7fab98005f90 comp rx=0 tx=0).stop 2024-02-26T06:27:56.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 msgr2=0x7faba81999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:56.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba81999e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab9402c0b0 tx=0x7fab94005d50 comp rx=0 tx=0).stop 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 shutdown_connections 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8106ea0 0x7faba81a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faba8104570 0x7faba81999e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fab8c0a21c0 0x7fab8c0a4680 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 --2- 172.21.15.6:0/840960605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faba8101c40 0x7faba81994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.910+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 >> 172.21.15.6:0/840960605 conn(0x7faba80fb650 msgr2=0x7faba81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.911+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 shutdown_connections 2024-02-26T06:27:56.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:56.911+0000 7fabadc00640 1 -- 172.21.15.6:0/840960605 wait complete. 2024-02-26T06:27:56.940 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-26T06:27:57.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.135+0000 7fdd76800640 1 -- 172.21.15.6:0/1851937279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701012b0 msgr2=0x7fdd70101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.135+0000 7fdd76800640 1 --2- 172.21.15.6:0/1851937279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701012b0 0x7fdd70101730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009900 tx=0x7fdd5802bc30 comp rx=0 tx=0).stop 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 -- 172.21.15.6:0/1851937279 shutdown_connections 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 --2- 172.21.15.6:0/1851937279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd70101c70 0x7fdd70108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 --2- 172.21.15.6:0/1851937279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701012b0 0x7fdd70101730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 --2- 172.21.15.6:0/1851937279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd701000c0 0x7fdd701004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 -- 172.21.15.6:0/1851937279 >> 172.21.15.6:0/1851937279 conn(0x7fdd700fb650 msgr2=0x7fdd700fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 -- 172.21.15.6:0/1851937279 shutdown_connections 2024-02-26T06:27:57.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.136+0000 7fdd76800640 1 -- 172.21.15.6:0/1851937279 wait complete. 2024-02-26T06:27:57.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 Processor -- start 2024-02-26T06:27:57.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 -- start start 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 0x7fdd701994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd701012b0 0x7fdd701999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd70101c70 0x7fdd701a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701a0f90 con 0x7fdd70101c70 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701a1100 con 0x7fdd701012b0 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.137+0000 7fdd76800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701a12c0 con 0x7fdd701000c0 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 0x7fdd701994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd66a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd701012b0 0x7fdd701999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 0x7fdd701994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:59056/0 (socket says 172.21.15.6:59056) 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 -- 172.21.15.6:0/3973180215 learned_addr learned my addr 172.21.15.6:0/3973180215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6fe00640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd70101c70 0x7fdd701a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 -- 172.21.15.6:0/3973180215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd701012b0 msgr2=0x7fdd701999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd701012b0 0x7fdd701999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 -- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd70101c70 msgr2=0x7fdd701a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd70101c70 0x7fdd701a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 -- 172.21.15.6:0/3973180215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009560 con 0x7fdd701000c0 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd66a00640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd701012b0 0x7fdd701999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.138+0000 7fdd6f400640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 0x7fdd701994a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000ebc0 tx=0x7fdd6000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.139+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60003b70 con 0x7fdd701000c0 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.139+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd6000eec0 con 0x7fdd701000c0 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.139+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60019800 con 0x7fdd701000c0 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.139+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd701a15a0 con 0x7fdd701000c0 2024-02-26T06:27:57.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.139+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd701a1ad0 con 0x7fdd701000c0 2024-02-26T06:27:57.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.140+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7004f730 con 0x7fdd701000c0 2024-02-26T06:27:57.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.142+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) v1 ==== 136835+0+0 (secure 0 0 0) 0x7fdd60019960 con 0x7fdd701000c0 2024-02-26T06:27:57.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.143+0000 7fdd6cc00640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 0x7fdd540a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.143+0000 7fdd66a00640 1 -- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 msgr2=0x7fdd540a46d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:57.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.143+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fdd60014070 con 0x7fdd701000c0 2024-02-26T06:27:57.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.143+0000 7fdd66a00640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 0x7fdd540a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:27:57.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-mon[36229]: scrub queued for path: / 2024-02-26T06:27:57.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:57.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-mon[36229]: scrub summary: active paths [/] 2024-02-26T06:27:57.147 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:57.147 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/840960605' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:27:57.147 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: ignoring --setuser ceph since I am not root 2024-02-26T06:27:57.147 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: ignoring --setgroup ceph since I am not root 2024-02-26T06:27:57.147 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:56 smithi167 ceph-mgr[38021]: -- 172.21.15.167:0/805949050 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d7865c1a0 con 0x560d78633000 2024-02-26T06:27:57.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.147+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd600974d0 con 0x7fdd701000c0 2024-02-26T06:27:57.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: ignoring --setuser ceph since I am not root 2024-02-26T06:27:57.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: ignoring --setgroup ceph since I am not root 2024-02-26T06:27:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:56 smithi006 ceph-mon[31790]: scrub queued for path: / 2024-02-26T06:27:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:56 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:56 smithi006 ceph-mon[31790]: scrub summary: active paths [/] 2024-02-26T06:27:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:56 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:57.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:56 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/840960605' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:27:57.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.344+0000 7fdd66a00640 1 -- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 msgr2=0x7fdd540a46d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:57.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.344+0000 7fdd66a00640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 0x7fdd540a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:27:57.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.354+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fdd7010bb20 con 0x7fdd701000c0 2024-02-26T06:27:57.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.363+0000 7fdd6cc00640 1 -- 172.21.15.6:0/3973180215 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdd6009ab90 con 0x7fdd701000c0 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-mon[35681]: scrub queued for path: / 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-mon[35681]: scrub summary: active paths [/] 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/840960605' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: ignoring --setuser ceph since I am not root 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: ignoring --setgroup ceph since I am not root 2024-02-26T06:27:57.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:56 smithi090 ceph-mgr[37112]: -- 172.21.15.90:0/1290532173 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561e63c8c1a0 con 0x561e63c62800 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.365+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 msgr2=0x7fdd540a46d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 0x7fdd540a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 msgr2=0x7fdd701994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 0x7fdd701994a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000ebc0 tx=0x7fdd6000c6a0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 shutdown_connections 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd70101c70 0x7fdd701a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7fdd540a2210 0x7fdd540a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd701012b0 0x7fdd701999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 --2- 172.21.15.6:0/3973180215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd701000c0 0x7fdd701994a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 >> 172.21.15.6:0/3973180215 conn(0x7fdd700fb650 msgr2=0x7fdd700fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:57.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 shutdown_connections 2024-02-26T06:27:57.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.366+0000 7fdd76800640 1 -- 172.21.15.6:0/3973180215 wait complete. 2024-02-26T06:27:57.386 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-26T06:27:57.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.596+0000 7f6635e00640 1 -- 172.21.15.6:0/3316757541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630107800 msgr2=0x7f6630107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.596+0000 7f6635e00640 1 --2- 172.21.15.6:0/3316757541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630107800 0x7f6630107c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6620009ca0 tx=0x7f662002c100 comp rx=0 tx=0).stop 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 -- 172.21.15.6:0/3316757541 shutdown_connections 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 --2- 172.21.15.6:0/3316757541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66301093b0 0x7f663010fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 --2- 172.21.15.6:0/3316757541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66301089f0 0x7f6630108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 --2- 172.21.15.6:0/3316757541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630107800 0x7f6630107c20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 -- 172.21.15.6:0/3316757541 >> 172.21.15.6:0/3316757541 conn(0x7f66300767d0 msgr2=0x7f6630078bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 -- 172.21.15.6:0/3316757541 shutdown_connections 2024-02-26T06:27:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.597+0000 7f6635e00640 1 -- 172.21.15.6:0/3316757541 wait complete. 2024-02-26T06:27:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 Processor -- start 2024-02-26T06:27:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 -- start start 2024-02-26T06:27:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 0x7f663019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66301089f0 0x7f663019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301093b0 0x7f66301a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301a5360 con 0x7f66301089f0 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301a54d0 con 0x7f6630107800 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.598+0000 7f6635e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301a5690 con 0x7f66301093b0 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 0x7f663019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 0x7f663019d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:47002/0 (socket says 172.21.15.6:47002) 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 -- 172.21.15.6:0/2931891899 learned_addr learned my addr 172.21.15.6:0/2931891899 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662e000640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66301089f0 0x7f663019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662f400640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301093b0 0x7f66301a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 -- 172.21.15.6:0/2931891899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301093b0 msgr2=0x7f66301a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301093b0 0x7f66301a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 -- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66301089f0 msgr2=0x7f663019de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66301089f0 0x7f663019de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 -- 172.21.15.6:0/2931891899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009900 con 0x7f6630107800 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.599+0000 7f662ea00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 0x7f663019d8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6620000f80 tx=0x7f6620032fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.600+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620034640 con 0x7f6630107800 2024-02-26T06:27:57.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.600+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6620003e50 con 0x7f6630107800 2024-02-26T06:27:57.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.600+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66301a5910 con 0x7f6630107800 2024-02-26T06:27:57.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.600+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620040450 con 0x7f6630107800 2024-02-26T06:27:57.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.600+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301a5de0 con 0x7f6630107800 2024-02-26T06:27:57.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.601+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663010eba0 con 0x7f6630107800 2024-02-26T06:27:57.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.603+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) v1 ==== 136835+0+0 (secure 0 0 0) 0x7f662000a040 con 0x7f6630107800 2024-02-26T06:27:57.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.604+0000 7f6634a00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 0x7f66140a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:57.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.605+0000 7f662e000640 1 -- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 msgr2=0x7f66140a46d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:57.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.605+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f66200f0690 con 0x7f6630107800 2024-02-26T06:27:57.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.605+0000 7f662e000640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 0x7f66140a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:27:57.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.608+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66200b95a0 con 0x7f6630107800 2024-02-26T06:27:57.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:57.482+0000 7f2cafb1f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:27:57.745 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:57.574+0000 7f2cafb1f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:27:57.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.806+0000 7f662e000640 1 -- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 msgr2=0x7f66140a46d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:57.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.806+0000 7f662e000640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 0x7f66140a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:27:57.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.808+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6630113320 con 0x7f6630107800 2024-02-26T06:27:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.816+0000 7f6634a00640 1 -- 172.21.15.6:0/2931891899 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f66200bcc60 con 0x7f6630107800 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 msgr2=0x7f66140a46d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 0x7f66140a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 msgr2=0x7f663019d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 0x7f663019d8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6620000f80 tx=0x7f6620032fa0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 shutdown_connections 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301093b0 0x7f66301a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66301089f0 0x7f663019de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f66140a2210 0x7f66140a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 --2- 172.21.15.6:0/2931891899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630107800 0x7f663019d8c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.819+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 >> 172.21.15.6:0/2931891899 conn(0x7f66300767d0 msgr2=0x7f663010bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.820+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 shutdown_connections 2024-02-26T06:27:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:57.820+0000 7f6635e00640 1 -- 172.21.15.6:0/2931891899 wait complete. 2024-02-26T06:27:57.838 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-26T06:27:57.865 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:57.480+0000 7fa510167200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:27:57.865 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:57.574+0000 7fa510167200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:27:58.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 -- 172.21.15.6:0/1721723575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c1043a0 msgr2=0x7f919c106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 --2- 172.21.15.6:0/1721723575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c1043a0 0x7f919c106790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9188009860 tx=0x7f918802bb70 comp rx=0 tx=0).stop 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 -- 172.21.15.6:0/1721723575 shutdown_connections 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 --2- 172.21.15.6:0/1721723575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f919c106cd0 0x7f919c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 --2- 172.21.15.6:0/1721723575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c1043a0 0x7f919c106790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 --2- 172.21.15.6:0/1721723575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919c101a70 0x7f919c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.043+0000 7f91a1e00640 1 -- 172.21.15.6:0/1721723575 >> 172.21.15.6:0/1721723575 conn(0x7f919c0fb440 msgr2=0x7f919c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:27:58.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.044+0000 7f91a1e00640 1 -- 172.21.15.6:0/1721723575 shutdown_connections 2024-02-26T06:27:58.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.044+0000 7f91a1e00640 1 -- 172.21.15.6:0/1721723575 wait complete. 2024-02-26T06:27:58.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.044+0000 7f91a1e00640 1 Processor -- start 2024-02-26T06:27:58.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 -- start start 2024-02-26T06:27:58.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101a70 0x7f919c1013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:58.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f919c1043a0 0x7f919c0ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:58.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919c106cd0 0x7f919c0fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:58.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1006c0 con 0x7f919c1043a0 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c100830 con 0x7f919c106cd0 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a10d0 con 0x7f919c101a70 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101a70 0x7f919c1013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101a70 0x7f919c1013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:59094/0 (socket says 172.21.15.6:59094) 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a0a00640 1 -- 172.21.15.6:0/3100000709 learned_addr learned my addr 172.21.15.6:0/3100000709 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f919c1043a0 0x7f919c0ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.045+0000 7f91a1400640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919c106cd0 0x7f919c0fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a0a00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919c106cd0 msgr2=0x7f919c0fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a0a00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919c106cd0 0x7f919c0fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:58.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a0a00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f919c1043a0 msgr2=0x7f919c0ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a0a00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f919c1043a0 0x7f919c0ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a0a00640 1 -- 172.21.15.6:0/3100000709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188009490 con 0x7f919c101a70 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a0a00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101a70 0x7f919c1013e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9c0 tx=0x7f919000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000ccc0 con 0x7f919c101a70 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f919000ce20 con 0x7f919c101a70 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c1a1380 con 0x7f919c101a70 2024-02-26T06:27:58.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190010430 con 0x7f919c101a70 2024-02-26T06:27:58.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.046+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c1a1860 con 0x7f919c101a70 2024-02-26T06:27:58.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.047+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c0fcf30 con 0x7f919c101a70 2024-02-26T06:27:58.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.049+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) v1 ==== 136835+0+0 (secure 0 0 0) 0x7f9190005000 con 0x7f919c101a70 2024-02-26T06:27:58.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.050+0000 7f9199600640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:27:58.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.051+0000 7f919be00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:58.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.051+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9190014070 con 0x7f919c101a70 2024-02-26T06:27:58.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.051+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T06:27:58.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.053+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9190096f30 con 0x7f919c101a70 2024-02-26T06:27:58.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:57 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/840960605' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-26T06:27:58.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:57 smithi167 ceph-mon[36229]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-26T06:27:58.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:57 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' 2024-02-26T06:27:58.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:27:57 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' 2024-02-26T06:27:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/840960605' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-26T06:27:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-mon[31790]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-26T06:27:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' 2024-02-26T06:27:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' 2024-02-26T06:27:58.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:57.776+0000 7f2436d68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T06:27:58.172 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:57.907+0000 7f2436d68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T06:27:58.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:57 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/840960605' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-26T06:27:58.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:57 smithi090 ceph-mon[35681]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-02-26T06:27:58.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:57 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' 2024-02-26T06:27:58.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:27:57 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' 2024-02-26T06:27:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.252+0000 7f919be00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.252+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T06:27:58.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.281+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 --> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f919c02ceb0 con 0x7f91840a2180 2024-02-26T06:27:58.429 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:58.219+0000 7f2cafb1f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:27:58.430 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:58.316+0000 7f2cafb1f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:27:58.543 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.212+0000 7fa510167200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:27:58.543 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.306+0000 7fa510167200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:27:58.543 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.447+0000 7fa510167200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:27:58.543 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.543+0000 7fa510167200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:27:58.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.653+0000 7f919be00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:58.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:58.653+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T06:27:58.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:58.464+0000 7f2cafb1f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:27:58.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:58.566+0000 7f2cafb1f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:27:58.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:58.654+0000 7f2cafb1f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:27:58.865 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.624+0000 7fa510167200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:27:59.048 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:58.736+0000 7f2436d68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T06:27:59.048 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:58.863+0000 7f2436d68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T06:27:59.048 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:59.047+0000 7f2436d68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T06:27:59.176 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.887+0000 7fa510167200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:27:59.176 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:58.995+0000 7fa510167200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:27:59.176 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:59.081+0000 7fa510167200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:27:59.176 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:59.176+0000 7fa510167200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:27:59.222 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:58.927+0000 7f2cafb1f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:27:59.222 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:59.039+0000 7f2cafb1f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:27:59.222 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:59.126+0000 7f2cafb1f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:27:59.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:59.177+0000 7f2436d68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T06:27:59.422 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:59.291+0000 7f2436d68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T06:27:59.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:59.454+0000 7f919be00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:27:59.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:27:59.454+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T06:27:59.494 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:59.221+0000 7f2cafb1f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:27:59.494 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:27:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:27:59.402+0000 7f2cafb1f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:27:59.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:27:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:27:59.350+0000 7fa510167200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:27:59.902 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:59.638+0000 7f2436d68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T06:27:59.903 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:59.783+0000 7f2436d68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T06:27:59.903 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:27:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:27:59.901+0000 7f2436d68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T06:28:00.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:00.031+0000 7f2436d68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T06:28:00.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:00.300+0000 7fa510167200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:28:00.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:00.429+0000 7fa510167200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:28:00.671 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:00.259+0000 7f2436d68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T06:28:00.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:00.375+0000 7f2cafb1f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:28:00.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:00.505+0000 7f2cafb1f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:28:01.037 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:00.758+0000 7fa510167200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:28:01.037 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:00.846+0000 7fa510167200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:28:01.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:01.056+0000 7f919be00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:28:01.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:01.056+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T06:28:01.117 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:00.836+0000 7f2cafb1f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:28:01.117 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:00.925+0000 7f2cafb1f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:28:01.117 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:01.116+0000 7f2cafb1f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:28:01.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:01.037+0000 7fa510167200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:28:01.365 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:01.219+0000 7fa510167200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:28:01.494 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:01.298+0000 7f2cafb1f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:28:01.758 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:01.502+0000 7fa510167200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:28:01.838 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:01.577+0000 7f2cafb1f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:28:01.921 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:01.587+0000 7f2436d68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T06:28:01.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:01.767+0000 7f2436d68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T06:28:02.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:01.758+0000 7fa510167200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:28:02.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:01.993+0000 7fa510167200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:28:02.244 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:01.838+0000 7f2cafb1f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:28:02.244 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:02.073+0000 7f2cafb1f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:28:02.596 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:02.204+0000 7f2436d68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T06:28:02.596 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:02.328+0000 7f2436d68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T06:28:02.856 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:02.595+0000 7f2436d68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:28:02.856 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:02.854+0000 7f2436d68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T06:28:03.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:02 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:02.786+0000 7fa510167200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:28:03.244 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:02.872+0000 7f2cafb1f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:28:03.579 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:03.248+0000 7f2436d68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T06:28:03.579 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:03.578+0000 7f2436d68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T06:28:03.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:03.351+0000 7fa510167200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:28:03.615 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:03.448+0000 7fa510167200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:28:03.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:03.446+0000 7f2cafb1f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:28:03.744 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:03.544+0000 7f2cafb1f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:28:03.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:03.744+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 21) v1 ==== 136835+0+0 (secure 0 0 0) 0x7f9190097300 con 0x7f919c101a70 2024-02-26T06:28:03.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-mon[31790]: Standby manager daemon y restarted 2024-02-26T06:28:03.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-mon[31790]: Standby manager daemon y started 2024-02-26T06:28:03.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:28:03.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:28:03.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:28:03.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:28:04.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y[37108]: 2024-02-26T06:28:03.660+0000 7fa510167200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:28:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-mon[35681]: Standby manager daemon y restarted 2024-02-26T06:28:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-mon[35681]: Standby manager daemon y started 2024-02-26T06:28:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:28:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:28:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:28:04.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:03 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:28:04.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:03.907+0000 7f2436d68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T06:28:04.245 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z[37979]: 2024-02-26T06:28:03.763+0000 7f2cafb1f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:28:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-mon[36229]: Standby manager daemon y restarted 2024-02-26T06:28:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-mon[36229]: Standby manager daemon y started 2024-02-26T06:28:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T06:28:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:28:04.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T06:28:04.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:03 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.90:0/3874590616' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:28:04.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:04.258+0000 7f919be00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/324687139 2024-02-26T06:28:04.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:04.258+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-26T06:28:04.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:04.775+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 8 ==== mgrmap(e 22) v1 ==== 136835+0+0 (secure 0 0 0) 0x7f91900cbcc0 con 0x7f919c101a70 2024-02-26T06:28:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-02-26T06:28:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: Standby manager daemon z restarted 2024-02-26T06:28:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: Standby manager daemon z started 2024-02-26T06:28:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:28:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:28:05.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:28:05.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:04 smithi090 ceph-mon[35681]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:28:05.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:04.998+0000 7f2436d68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: Standby manager daemon z restarted 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: Standby manager daemon z started 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:28:05.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:04 smithi006 ceph-mon[31790]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:28:05.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-02-26T06:28:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: Standby manager daemon z restarted 2024-02-26T06:28:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: Standby manager daemon z started 2024-02-26T06:28:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T06:28:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T06:28:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T06:28:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:04 smithi167 ceph-mon[36229]: from='mgr.? 172.21.15.167:0/2333928835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T06:28:06.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:05 smithi090 ceph-mon[35681]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-02-26T06:28:06.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:05 smithi006 ceph-mon[31790]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-02-26T06:28:06.170 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:05.730+0000 7f2436d68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T06:28:06.170 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:05.870+0000 7f2436d68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T06:28:06.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:06.196+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 9 ==== mgrmap(e 23) v1 ==== 136453+0+0 (secure 0 0 0) 0x7f91900052b0 con 0x7f919c101a70 2024-02-26T06:28:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:06.198+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 msgr2=0x7f91840a4640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T06:28:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:06.198+0000 7f9199600640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/324687139,v1:172.21.15.6:6801/324687139] conn(0x7f91840a2180 0x7f91840a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:06.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:05 smithi167 ceph-mon[36229]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-02-26T06:28:06.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.169+0000 7f2436d68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T06:28:06.768 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.491+0000 7f238d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.768 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.491+0000 7f238d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.768 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.491+0000 7f238d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.768 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.491+0000 7f238d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.769 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.491+0000 7f238d200700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.769 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.492+0000 7f2388c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.769 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.492+0000 7f2388c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.769 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.492+0000 7f2388c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.769 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.492+0000 7f2388c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:06.769 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:28:06.492+0000 7f2388c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:28:07.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: scrub summary: idle 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: Active manager daemon x restarted 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: Activating manager daemon x 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: mgrmap e23: x(active, starting, since 0.0165777s), standbys: z, y 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:28:07.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:28:07.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: Manager daemon x is now available 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:28:07.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:06 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:28:07.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: scrub summary: idle 2024-02-26T06:28:07.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: Active manager daemon x restarted 2024-02-26T06:28:07.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: Activating manager daemon x 2024-02-26T06:28:07.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T06:28:07.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: mgrmap e23: x(active, starting, since 0.0165777s), standbys: z, y 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:28:07.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:28:07.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: Manager daemon x is now available 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:28:07.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:28:07.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:28:07.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:06 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:28:07.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.207+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mon.1 v2:172.21.15.167:3300/0 10 ==== mgrmap(e 24) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f9190097bd0 con 0x7f919c101a70 2024-02-26T06:28:07.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.208+0000 7f9199600640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f91840674a0 0x7f9184067880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:07.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.208+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 --> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f919c02ceb0 con 0x7f91840674a0 2024-02-26T06:28:07.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.217+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f91840674a0 0x7f9184067880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:07.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.225+0000 7f919be00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f91840674a0 0x7f9184067880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9188034b10 tx=0x7f9188034be0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: scrub summary: idle 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: Active manager daemon x restarted 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: Activating manager daemon x 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: mgrmap e23: x(active, starting, since 0.0165777s), standbys: z, y 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:28:07.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T06:28:07.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T06:28:07.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: Manager daemon x is now available 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:28:07.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:06 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T06:28:07.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.321+0000 7f9199600640 1 -- 172.21.15.6:0/3100000709 <== mgr.15330 v2:172.21.15.6:6800/3331986569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f919c02ceb0 con 0x7f91840674a0 2024-02-26T06:28:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:Schedule set for path / 2024-02-26T06:28:07.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f91840674a0 msgr2=0x7f9184067880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f91840674a0 0x7f9184067880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9188034b10 tx=0x7f9188034be0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101a70 msgr2=0x7f919c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 --2- 172.21.15.6:0/3100000709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101a70 0x7f919c1013e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9c0 tx=0x7f919000ee80 comp rx=0 tx=0).stop 2024-02-26T06:28:07.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1400640 1 -- 172.21.15.6:0/3100000709 reap_dead start 2024-02-26T06:28:07.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 shutdown_connections 2024-02-26T06:28:07.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 >> 172.21.15.6:0/3100000709 conn(0x7f919c0fb440 msgr2=0x7f919c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:07.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 shutdown_connections 2024-02-26T06:28:07.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.324+0000 7f91a1e00640 1 -- 172.21.15.6:0/3100000709 wait complete. 2024-02-26T06:28:07.341 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 -- 172.21.15.6:0/3975550955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1300ff120 msgr2=0x7fc1300ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 --2- 172.21.15.6:0/3975550955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1300ff120 0x7fc1300ff540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc120009900 tx=0x7fc12002bc50 comp rx=0 tx=0).stop 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 -- 172.21.15.6:0/3975550955 shutdown_connections 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 --2- 172.21.15.6:0/3975550955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc130100b30 0x7fc1301013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 --2- 172.21.15.6:0/3975550955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc130100190 0x7fc1301005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 --2- 172.21.15.6:0/3975550955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1300ff120 0x7fc1300ff540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 -- 172.21.15.6:0/3975550955 >> 172.21.15.6:0/3975550955 conn(0x7fc1300fa960 msgr2=0x7fc1300fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 -- 172.21.15.6:0/3975550955 shutdown_connections 2024-02-26T06:28:07.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.546+0000 7fc135000640 1 -- 172.21.15.6:0/3975550955 wait complete. 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.547+0000 7fc135000640 1 Processor -- start 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 -- start start 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 0x7fc1301994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc130100190 0x7fc130199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc130100b30 0x7fc1301a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301a0fb0 con 0x7fc130100b30 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301a1120 con 0x7fc130100190 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc135000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301a12e0 con 0x7fc1300ff120 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc12de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 0x7fc1301994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc12e800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc130100b30 0x7fc1301a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc12de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 0x7fc1301994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58028/0 (socket says 172.21.15.6:58028) 2024-02-26T06:28:07.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc12de00640 1 -- 172.21.15.6:0/3773752425 learned_addr learned my addr 172.21.15.6:0/3773752425 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.548+0000 7fc125400640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc130100190 0x7fc130199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12de00640 1 -- 172.21.15.6:0/3773752425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc130100190 msgr2=0x7fc130199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12de00640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc130100190 0x7fc130199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12de00640 1 -- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc130100b30 msgr2=0x7fc1301a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12de00640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc130100b30 0x7fc1301a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12de00640 1 -- 172.21.15.6:0/3773752425 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128009840 con 0x7fc1300ff120 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12e800640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc130100b30 0x7fc1301a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc12de00640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 0x7fc1301994c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc12002c6c0 tx=0x7fc1200040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:07.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1200044c0 con 0x7fc1300ff120 2024-02-26T06:28:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc120009560 con 0x7fc1300ff120 2024-02-26T06:28:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.549+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1200052d0 con 0x7fc1300ff120 2024-02-26T06:28:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.550+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120003e10 con 0x7fc1300ff120 2024-02-26T06:28:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.550+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1301a18e0 con 0x7fc1300ff120 2024-02-26T06:28:07.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.551+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13010b650 con 0x7fc1300ff120 2024-02-26T06:28:07.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.553+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fc120005440 con 0x7fc1300ff120 2024-02-26T06:28:07.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.554+0000 7fc127400640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc1100a1ea0 0x7fc1100a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:07.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.554+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fc1200f03d0 con 0x7fc1300ff120 2024-02-26T06:28:07.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.554+0000 7fc125400640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc1100a1ea0 0x7fc1100a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:07.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.555+0000 7fc125400640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc1100a1ea0 0x7fc1100a4360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc13019a9d0 tx=0x7fc118005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:07.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.556+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1200b9510 con 0x7fc1300ff120 2024-02-26T06:28:07.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.818+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 --> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc13002ce40 con 0x7fc1100a1ea0 2024-02-26T06:28:07.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.965+0000 7fc127400640 1 -- 172.21.15.6:0/3773752425 <== mgr.15330 v2:172.21.15.6:6800/3331986569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fc13002ce40 con 0x7fc1100a1ea0 2024-02-26T06:28:07.966 INFO:teuthology.orchestra.run.smithi006.stdout:Retention added to path / 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc1100a1ea0 msgr2=0x7fc1100a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc1100a1ea0 0x7fc1100a4360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc13019a9d0 tx=0x7fc118005f90 comp rx=0 tx=0).stop 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 msgr2=0x7fc1301994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 0x7fc1301994c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc12002c6c0 tx=0x7fc1200040c0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 shutdown_connections 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc1100a1ea0 0x7fc1100a4360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc130100b30 0x7fc1301a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc130100190 0x7fc130199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 --2- 172.21.15.6:0/3773752425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1300ff120 0x7fc1301994c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 >> 172.21.15.6:0/3773752425 conn(0x7fc1300fa960 msgr2=0x7fc1300fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 shutdown_connections 2024-02-26T06:28:07.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:07.970+0000 7fc135000640 1 -- 172.21.15.6:0/3773752425 wait complete. 2024-02-26T06:28:07.991 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-26T06:28:08.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: [26/Feb/2024:06:28:06] ENGINE Bus STARTING 2024-02-26T06:28:08.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: [26/Feb/2024:06:28:06] ENGINE Serving on https://172.21.15.6:7150 2024-02-26T06:28:08.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: [26/Feb/2024:06:28:07] ENGINE Serving on http://172.21.15.6:8765 2024-02-26T06:28:08.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: [26/Feb/2024:06:28:07] ENGINE Bus STARTED 2024-02-26T06:28:08.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: mgrmap e24: x(active, since 1.03704s), standbys: z, y 2024-02-26T06:28:08.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:08.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:07 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: [26/Feb/2024:06:28:06] ENGINE Bus STARTING 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: [26/Feb/2024:06:28:06] ENGINE Serving on https://172.21.15.6:7150 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: [26/Feb/2024:06:28:07] ENGINE Serving on http://172.21.15.6:8765 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: [26/Feb/2024:06:28:07] ENGINE Bus STARTED 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: mgrmap e24: x(active, since 1.03704s), standbys: z, y 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:08.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:07 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: [26/Feb/2024:06:28:06] ENGINE Bus STARTING 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: [26/Feb/2024:06:28:06] ENGINE Serving on https://172.21.15.6:7150 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: [26/Feb/2024:06:28:07] ENGINE Serving on http://172.21.15.6:8765 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: [26/Feb/2024:06:28:07] ENGINE Bus STARTED 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: mgrmap e24: x(active, since 1.03704s), standbys: z, y 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:08.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:07 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:08.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 -- 172.21.15.6:0/3117434409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741011d0 msgr2=0x7f64741015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 --2- 172.21.15.6:0/3117434409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741011d0 0x7f64741015f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6468009900 tx=0x7f646802bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:08.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 -- 172.21.15.6:0/3117434409 shutdown_connections 2024-02-26T06:28:08.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 --2- 172.21.15.6:0/3117434409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6474102d80 0x7f6474109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 --2- 172.21.15.6:0/3117434409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741023c0 0x7f6474102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 --2- 172.21.15.6:0/3117434409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741011d0 0x7f64741015f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 -- 172.21.15.6:0/3117434409 >> 172.21.15.6:0/3117434409 conn(0x7f64740fc740 msgr2=0x7f64740feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:08.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 -- 172.21.15.6:0/3117434409 shutdown_connections 2024-02-26T06:28:08.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.206+0000 7f647be00640 1 -- 172.21.15.6:0/3117434409 wait complete. 2024-02-26T06:28:08.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.207+0000 7f647be00640 1 Processor -- start 2024-02-26T06:28:08.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.207+0000 7f647be00640 1 -- start start 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.207+0000 7f647be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 0x7f64741070c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.207+0000 7f647be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741023c0 0x7f6474107600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f647be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6474102d80 0x7f6474103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f647be00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741043c0 con 0x7f64741011d0 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f647be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474104530 con 0x7f6474102d80 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f647be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741046f0 con 0x7f64741023c0 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 0x7f64741070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 0x7f64741070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37134/0 (socket says 172.21.15.6:37134) 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 -- 172.21.15.6:0/3126421544 learned_addr learned my addr 172.21.15.6:0/3126421544 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6478a00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741023c0 0x7f6474107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479e00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6474102d80 0x7f6474103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 -- 172.21.15.6:0/3126421544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741023c0 msgr2=0x7f6474107600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741023c0 0x7f6474107600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 -- 172.21.15.6:0/3126421544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6474102d80 msgr2=0x7f6474103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6474102d80 0x7f6474103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6479400640 1 -- 172.21.15.6:0/3126421544 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6468009560 con 0x7f64741011d0 2024-02-26T06:28:08.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.208+0000 7f6478a00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741023c0 0x7f6474107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:08.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.209+0000 7f6479400640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 0x7f64741070c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6468000c00 tx=0x7f64680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:08.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.209+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64680044c0 con 0x7f64741011d0 2024-02-26T06:28:08.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.209+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64741a5970 con 0x7f64741011d0 2024-02-26T06:28:08.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.209+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64680052d0 con 0x7f64741011d0 2024-02-26T06:28:08.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.209+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646803e450 con 0x7f64741011d0 2024-02-26T06:28:08.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.209+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64741a5e80 con 0x7f64741011d0 2024-02-26T06:28:08.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.210+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647410cc30 con 0x7f64741011d0 2024-02-26T06:28:08.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.212+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 24) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f6468005440 con 0x7f64741011d0 2024-02-26T06:28:08.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.214+0000 7f6472000640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f645c0a1ea0 0x7f645c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.214+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f64680f01d0 con 0x7f64741011d0 2024-02-26T06:28:08.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.214+0000 7f6478a00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f645c0a1ea0 0x7f645c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.215+0000 7f6478a00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f645c0a1ea0 0x7f645c0a4360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6474105040 tx=0x7f646000a2d0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:08.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.218+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64680b9470 con 0x7f64741011d0 2024-02-26T06:28:08.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.451+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 --> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f6474109280 con 0x7f645c0a1ea0 2024-02-26T06:28:08.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.467+0000 7f6472000640 1 -- 172.21.15.6:0/3126421544 <== mgr.15330 v2:172.21.15.6:6800/3331986569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f6474109280 con 0x7f645c0a1ea0 2024-02-26T06:28:08.468 INFO:teuthology.orchestra.run.smithi006.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-26T00:00:00", "created": "2024-02-26T06:28:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f645c0a1ea0 msgr2=0x7f645c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f645c0a1ea0 0x7f645c0a4360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6474105040 tx=0x7f646000a2d0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 msgr2=0x7f64741070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 0x7f64741070c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6468000c00 tx=0x7f64680040c0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 shutdown_connections 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f645c0a1ea0 0x7f645c0a4360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6474102d80 0x7f6474103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64741023c0 0x7f6474107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 --2- 172.21.15.6:0/3126421544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64741011d0 0x7f64741070c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 >> 172.21.15.6:0/3126421544 conn(0x7f64740fc740 msgr2=0x7f6474107b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:08.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 shutdown_connections 2024-02-26T06:28:08.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.471+0000 7f647be00640 1 -- 172.21.15.6:0/3126421544 wait complete. 2024-02-26T06:28:08.490 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-26T06:28:08.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.705+0000 7fd485e00640 1 -- 172.21.15.6:0/22093643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480073b60 msgr2=0x7fd480073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.705+0000 7fd485e00640 1 --2- 172.21.15.6:0/22093643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480073b60 0x7fd480073fc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd468009900 tx=0x7fd46802bcc0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 -- 172.21.15.6:0/22093643 shutdown_connections 2024-02-26T06:28:08.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 --2- 172.21.15.6:0/22093643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480109080 0x7fd48010b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 --2- 172.21.15.6:0/22093643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480073b60 0x7fd480073fc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 --2- 172.21.15.6:0/22093643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 0x7fd480073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 -- 172.21.15.6:0/22093643 >> 172.21.15.6:0/22093643 conn(0x7fd4800fc740 msgr2=0x7fd4800feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:08.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 -- 172.21.15.6:0/22093643 shutdown_connections 2024-02-26T06:28:08.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.706+0000 7fd485e00640 1 -- 172.21.15.6:0/22093643 wait complete. 2024-02-26T06:28:08.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 Processor -- start 2024-02-26T06:28:08.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 -- start start 2024-02-26T06:28:08.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480073b60 0x7fd480199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 0x7fd4801999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480109080 0x7fd4801a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801a0f70 con 0x7fd480073b60 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801a10e0 con 0x7fd480075140 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd485e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801a12a0 con 0x7fd480109080 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.707+0000 7fd47ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480073b60 0x7fd480199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 0x7fd4801999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd47ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480073b60 0x7fd480199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37172/0 (socket says 172.21.15.6:37172) 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd47ea00640 1 -- 172.21.15.6:0/1463346521 learned_addr learned my addr 172.21.15.6:0/1463346521 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd47f400640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480109080 0x7fd4801a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 -- 172.21.15.6:0/1463346521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480109080 msgr2=0x7fd4801a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480109080 0x7fd4801a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 -- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480073b60 msgr2=0x7fd480199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:08.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480073b60 0x7fd480199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:08.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 -- 172.21.15.6:0/1463346521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478009840 con 0x7fd480075140 2024-02-26T06:28:08.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.708+0000 7fd477e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 0x7fd4801999c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd46802c200 tx=0x7fd468005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:08.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.709+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46802dc10 con 0x7fd480075140 2024-02-26T06:28:08.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.709+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd46802dd70 con 0x7fd480075140 2024-02-26T06:28:08.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.709+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468003af0 con 0x7fd480075140 2024-02-26T06:28:08.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.709+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd468009560 con 0x7fd480075140 2024-02-26T06:28:08.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.709+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4801a1820 con 0x7fd480075140 2024-02-26T06:28:08.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.710+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd480107e40 con 0x7fd480075140 2024-02-26T06:28:08.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.713+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fd468005440 con 0x7fd480075140 2024-02-26T06:28:08.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.713+0000 7fd484a00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fd4640a1f20 0x7fd4640a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:08.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.714+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fd4680eff20 con 0x7fd480075140 2024-02-26T06:28:08.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.714+0000 7fd47ea00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fd4640a1f20 0x7fd4640a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:08.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.715+0000 7fd47ea00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fd4640a1f20 0x7fd4640a43e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd470009cd0 tx=0x7fd470005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:08.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.717+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4680b90f0 con 0x7fd480075140 2024-02-26T06:28:08.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.791+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fd4680b94c0 con 0x7fd480075140 2024-02-26T06:28:08.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.985+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 --> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd48002ceb0 con 0x7fd4640a1f20 2024-02-26T06:28:08.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:08.998+0000 7fd484a00640 1 -- 172.21.15.6:0/1463346521 <== mgr.15330 v2:172.21.15.6:6800/3331986569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd48002ceb0 con 0x7fd4640a1f20 2024-02-26T06:28:08.998 INFO:teuthology.orchestra.run.smithi006.stdout:/ 1m 6m3h 2024-02-26T06:28:09.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fd4640a1f20 msgr2=0x7fd4640a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:09.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fd4640a1f20 0x7fd4640a43e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd470009cd0 tx=0x7fd470005f90 comp rx=0 tx=0).stop 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 msgr2=0x7fd4801999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 0x7fd4801999c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd46802c200 tx=0x7fd468005d70 comp rx=0 tx=0).stop 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 shutdown_connections 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fd4640a1f20 0x7fd4640a43e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd480109080 0x7fd4801a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd480075140 0x7fd4801999c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 --2- 172.21.15.6:0/1463346521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd480073b60 0x7fd480199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 >> 172.21.15.6:0/1463346521 conn(0x7fd4800fc740 msgr2=0x7fd480105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 shutdown_connections 2024-02-26T06:28:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:09.001+0000 7fd485e00640 1 -- 172.21.15.6:0/1463346521 wait complete. 2024-02-26T06:28:09.021 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-26T06:28:09.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:08 smithi006 ceph-mon[31790]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:09.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:08 smithi006 ceph-mon[31790]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:09.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:08 smithi006 ceph-mon[31790]: from='client.25501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:09.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:08 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:09.086 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:08 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:09.099 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-26T06:28:09.110 INFO:teuthology.run_tasks:Running task background_exec... 2024-02-26T06:28:09.158 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:28:09.159 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-02-26T06:28:09.159 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:08 smithi090 ceph-mon[35681]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:09.160 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:08 smithi090 ceph-mon[35681]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:09.160 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:08 smithi090 ceph-mon[35681]: from='client.25501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:09.160 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:08 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:09.160 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:08 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:09.165 INFO:teuthology.run_tasks:Running task workunit... 2024-02-26T06:28:09.193 INFO:tasks.workunit:Pulling workunits from ref ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 2024-02-26T06:28:09.194 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-26T06:28:09.194 INFO:tasks.workunit:timeout=3h 2024-02-26T06:28:09.194 INFO:tasks.workunit:cleanup=True 2024-02-26T06:28:09.194 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-26T06:28:09.241 INFO:teuthology.orchestra.run.smithi006.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-26T06:28:09.241 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-02-26 06:25:25.311995825 +0000 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-02-26 06:25:25.311995825 +0000 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-02-26 06:27:44.988523721 +0000 2024-02-26T06:28:09.242 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-02-26 06:25:25.311995825 +0000 2024-02-26T06:28:09.242 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-26T06:28:09.242 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-26T06:28:09.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:08 smithi167 ceph-mon[36229]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:09.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:08 smithi167 ceph-mon[36229]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:09.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:08 smithi167 ceph-mon[36229]: from='client.25501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:09.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:08 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:09.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:08 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:09.383 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/vshankar/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 2024-02-26T06:28:09.426 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: mgrmap e25: x(active, since 2s), standbys: z, y 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='client.25535 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:10.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:09 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:10.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: mgrmap e25: x(active, since 2s), standbys: z, y 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='client.25535 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:09 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: mgrmap e25: x(active, since 2s), standbys: z, y 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='client.25535 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:10.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:09 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi090 to 174.1M 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: Unable to set osd_memory_target on smithi090 to 182612684: error parsing value: Value '182612684' is below minimum 939524096 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:10 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi090 to 174.1M 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: Unable to set osd_memory_target on smithi090 to 182612684: error parsing value: Value '182612684' is below minimum 939524096 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:10 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi090 to 174.1M 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: Unable to set osd_memory_target on smithi090 to 182612684: error parsing value: Value '182612684' is below minimum 939524096 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:11.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:10 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi167 to 174.1M 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: Unable to set osd_memory_target on smithi167 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: mgrmap e26: x(active, since 4s), standbys: z, y 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.001 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.001 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.001 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:12.001 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:11 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:12.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi167 to 174.1M 2024-02-26T06:28:12.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: Unable to set osd_memory_target on smithi167 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-02-26T06:28:12.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: mgrmap e26: x(active, since 4s), standbys: z, y 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:12.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:11 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:12.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi167 to 174.1M 2024-02-26T06:28:12.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: Unable to set osd_memory_target on smithi167 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: mgrmap e26: x(active, since 4s), standbys: z, y 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:28:12.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:12.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:11 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi006 to 177.8M 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Unable to set osd_memory_target on smithi006 to 186497024: error parsing value: Value '186497024' is below minimum 939524096 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Updating smithi006:/etc/ceph/ceph.conf 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Updating smithi090:/etc/ceph/ceph.conf 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Updating smithi167:/etc/ceph/ceph.conf 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:13.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:12 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:13.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi006 to 177.8M 2024-02-26T06:28:13.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Unable to set osd_memory_target on smithi006 to 186497024: error parsing value: Value '186497024' is below minimum 939524096 2024-02-26T06:28:13.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Updating smithi006:/etc/ceph/ceph.conf 2024-02-26T06:28:13.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Updating smithi090:/etc/ceph/ceph.conf 2024-02-26T06:28:13.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Updating smithi167:/etc/ceph/ceph.conf 2024-02-26T06:28:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:13.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:12 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:13.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi006 to 177.8M 2024-02-26T06:28:13.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Unable to set osd_memory_target on smithi006 to 186497024: error parsing value: Value '186497024' is below minimum 939524096 2024-02-26T06:28:13.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Updating smithi006:/etc/ceph/ceph.conf 2024-02-26T06:28:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Updating smithi090:/etc/ceph/ceph.conf 2024-02-26T06:28:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Updating smithi167:/etc/ceph/ceph.conf 2024-02-26T06:28:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:13.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:12 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:14.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:14.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:13 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:14.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:13 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:14.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T06:28:14.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:14.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T06:28:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: Updating smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: Updating smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:14.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:13 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:15.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:15.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:14 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:15.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:14 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-02-26T06:28:15.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: Updating smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:28:15.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:14 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:28:16.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:15 smithi090 ceph-mon[35681]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-02-26T06:28:16.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:15 smithi006 ceph-mon[31790]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-02-26T06:28:16.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:15 smithi167 ceph-mon[36229]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-02-26T06:28:18.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:17 smithi090 ceph-mon[35681]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-02-26T06:28:18.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:17 smithi006 ceph-mon[31790]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-02-26T06:28:18.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:17 smithi167 ceph-mon[36229]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-02-26T06:28:20.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:19 smithi090 ceph-mon[35681]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-02-26T06:28:20.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:19 smithi006 ceph-mon[31790]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-02-26T06:28:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:19 smithi167 ceph-mon[36229]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-02-26T06:28:22.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:21 smithi090 ceph-mon[35681]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-02-26T06:28:22.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:21 smithi006 ceph-mon[31790]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-02-26T06:28:22.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:21 smithi167 ceph-mon[36229]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-02-26T06:28:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 -- 172.21.15.6:0/3562914867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920069790 msgr2=0x7ff920069c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 --2- 172.21.15.6:0/3562914867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920069790 0x7ff920069c10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009930 tx=0x7ff90c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 -- 172.21.15.6:0/3562914867 shutdown_connections 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 --2- 172.21.15.6:0/3562914867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff92006a150 0x7ff92010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 --2- 172.21.15.6:0/3562914867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920069790 0x7ff920069c10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 --2- 172.21.15.6:0/3562914867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920106150 0x7ff920106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 -- 172.21.15.6:0/3562914867 >> 172.21.15.6:0/3562914867 conn(0x7ff920076e40 msgr2=0x7ff920077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 -- 172.21.15.6:0/3562914867 shutdown_connections 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.407+0000 7ff926800640 1 -- 172.21.15.6:0/3562914867 wait complete. 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.408+0000 7ff926800640 1 Processor -- start 2024-02-26T06:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 -- start start 2024-02-26T06:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 0x7ff9201003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff92006a150 0x7ff9201008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920106150 0x7ff920100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a5bb0 con 0x7ff92006a150 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a5d20 con 0x7ff920106150 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff926800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a5ee0 con 0x7ff920069790 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.409+0000 7ff91f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 0x7ff9201003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920106150 0x7ff920100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 0x7ff9201003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:38580/0 (socket says 172.21.15.6:38580) 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91f400640 1 -- 172.21.15.6:0/3640558969 learned_addr learned my addr 172.21.15.6:0/3640558969 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff92006a150 0x7ff9201008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 -- 172.21.15.6:0/3640558969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 msgr2=0x7ff9201003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 0x7ff9201003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 -- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff92006a150 msgr2=0x7ff9201008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff92006a150 0x7ff9201008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 -- 172.21.15.6:0/3640558969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c009560 con 0x7ff920106150 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91f400640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 0x7ff9201003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.410+0000 7ff91fe00640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920106150 0x7ff920100e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff91800da10 tx=0x7ff91800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.411+0000 7ff917e00640 1 -- 172.21.15.6:0/3640558969 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800bd70 con 0x7ff920106150 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.411+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9201a61c0 con 0x7ff920106150 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.411+0000 7ff917e00640 1 -- 172.21.15.6:0/3640558969 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff918003e50 con 0x7ff920106150 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.411+0000 7ff917e00640 1 -- 172.21.15.6:0/3640558969 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180106c0 con 0x7ff920106150 2024-02-26T06:28:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.411+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201a66a0 con 0x7ff920106150 2024-02-26T06:28:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.414+0000 7ff92977a740 1 Processor -- start 2024-02-26T06:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff917e00640 1 -- 172.21.15.6:0/3640558969 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7ff918004000 con 0x7ff920106150 2024-02-26T06:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 -- start start 2024-02-26T06:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e7e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323e2e5f0 con 0x561323e73af0 2024-02-26T06:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323de0890 con 0x561323e72d80 2024-02-26T06:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.416+0000 7ff92977a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323ddddc0 con 0x561323e7c300 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e72840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55462/0 (socket says 172.21.15.6:55462) 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e73200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50852/0 (socket says 172.21.15.6:50852) 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91ea00640 1 -- 172.21.15.6:0/2279395591 learned_addr learned my addr 172.21.15.6:0/2279395591 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91ea00640 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e7e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 -- 172.21.15.6:0/2279395591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 msgr2=0x561323e7e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e7e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 -- 172.21.15.6:0/2279395591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 msgr2=0x561323e72840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e72840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 -- 172.21.15.6:0/2279395591 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561323d5aad0 con 0x561323e72d80 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91ea00640 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e7e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.417+0000 7ff91f400640 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e73200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff910010480 tx=0x7ff91000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d63ac317fef87cb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.418+0000 7ff917e00640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff9040a22b0 0x7ff9040a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.418+0000 7ff914c00640 1 -- 172.21.15.6:0/2279395591 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9100117c0 con 0x561323e72d80 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.418+0000 7ff914c00640 1 -- 172.21.15.6:0/2279395591 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff910015070 con 0x561323e72d80 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.418+0000 7ff914c00640 1 -- 172.21.15.6:0/2279395591 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91001cde0 con 0x561323e72d80 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.418+0000 7ff917e00640 1 -- 172.21.15.6:0/3640558969 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7ff9180ce5e0 con 0x7ff920106150 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.418+0000 7ff91ea00640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff9040a22b0 0x7ff9040a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 -- 172.21.15.6:0/2279395591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 msgr2=0x561323e73200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e73200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff910010480 tx=0x7ff91000bf90 comp rx=0 tx=0).stop 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 -- 172.21.15.6:0/2279395591 shutdown_connections 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e7e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e73200 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 --2- 172.21.15.6:0/2279395591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e72840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 -- 172.21.15.6:0/2279395591 >> 172.21.15.6:0/2279395591 conn(0x561323e3a480 msgr2=0x561323e6da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff91ea00640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff9040a22b0 0x7ff9040a4770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005e70 tx=0x7ff90c005e00 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 -- 172.21.15.6:0/2279395591 shutdown_connections 2024-02-26T06:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 -- 172.21.15.6:0/2279395591 wait complete. 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.419+0000 7ff92977a740 1 -- start start 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 Processor -- start 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323dd95a0 con 0x561323e73af0 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323d7eb20 con 0x561323e72d80 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323d7e7c0 con 0x561323e7c300 2024-02-26T06:28:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.420+0000 7ff92977a740 10 client.0 _ll_register_callbacks cb 0x561323d92520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e792b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50854/0 (socket says 172.21.15.6:50854) 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 -- 172.21.15.6:0/1413593837 learned_addr learned my addr 172.21.15.6:0/1413593837 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91fe00640 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 -- 172.21.15.6:0/1413593837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 msgr2=0x561323e79d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e79d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 -- 172.21.15.6:0/1413593837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 msgr2=0x561323e797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e797f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 -- 172.21.15.6:0/1413593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91000bc70 con 0x561323e72d80 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff91f400640 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e792b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b6c0 tx=0x7ff91001c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37e6c3373ecd9a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.421+0000 7ff909600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff909600640 1 -- 172.21.15.6:0/1413593837 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910020040 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561323dabca0 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff909600640 1 -- 172.21.15.6:0/1413593837 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7ff910017070 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff909600640 1 -- 172.21.15.6:0/1413593837 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff91001dec0 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff92977a740 10 client.25547 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561323dd7c20 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.422+0000 7ff909600640 1 -- 172.21.15.6:0/1413593837 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff910033020 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff909600640 1 -- 172.21.15.6:0/1413593837 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff91002eaf0 con 0x561323e72d80 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff92977a740 10 client.25547 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff92977a740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff92977a740 20 client.25547 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff92977a740 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323e8c180 0x561323e8e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff92977a740 4 client.25547 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561323dcfcb0 con 0x561323e8c180 2024-02-26T06:28:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.423+0000 7ff91ea00640 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323e8c180 0x561323e8e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.424+0000 7ff91ea00640 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323e8c180 0x561323e8e570 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c724c1ecdfdf709f server_cookie=1556aba304e25da8 in_seq=0 out_seq=0 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.424+0000 7ff909600640 10 client.25547 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.424+0000 7ff909600640 1 -- 172.21.15.6:0/1413593837 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561323dcfcb0 con 0x561323e8c180 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.424+0000 7ff909600640 10 client.25547 handle_command_reply: tid=0 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.424+0000 7ff92977a740 1 client.25547 shutdown 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 msgr2=0x561323e792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e792b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b6c0 tx=0x7ff91001c720 comp rx=0 tx=0).stop 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 shutdown_connections 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323e8c180 0x561323e8e570 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e7c300 0x561323e79d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e73af0 0x561323e797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff909600640 0 client.25547 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 --2- 172.21.15.6:0/1413593837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e792b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 >> 172.21.15.6:0/1413593837 conn(0x561323e3a480 msgr2=0x561323e6f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 shutdown_connections 2024-02-26T06:28:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 1 -- 172.21.15.6:0/1413593837 wait complete. 2024-02-26T06:28:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.425+0000 7ff92977a740 20 client.25547 trim_cache size 0 max 16384 2024-02-26T06:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.439+0000 7ff92977a740 1 Processor -- start 2024-02-26T06:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.439+0000 7ff92977a740 1 -- start start 2024-02-26T06:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.439+0000 7ff92977a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.439+0000 7ff92977a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e73af0 0x561323e79b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.439+0000 7ff92977a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e83120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff92977a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323b21e60 con 0x561323e7c300 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff92977a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323e8bf80 con 0x561323e72d80 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff92977a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323e740e0 con 0x561323e73af0 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e83120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e83120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55492/0 (socket says 172.21.15.6:55492) 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91fe00640 1 -- 172.21.15.6:0/3786135071 learned_addr learned my addr 172.21.15.6:0/3786135071 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91ea00640 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e73af0 0x561323e79b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91f400640 1 -- 172.21.15.6:0/3786135071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e73af0 msgr2=0x561323e79b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91f400640 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e73af0 0x561323e79b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91f400640 1 -- 172.21.15.6:0/3786135071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 msgr2=0x561323e83120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91f400640 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e83120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91f400640 1 -- 172.21.15.6:0/3786135071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561323d74500 con 0x561323e72d80 2024-02-26T06:28:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91ea00640 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e73af0 0x561323e79b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.440+0000 7ff91fe00640 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e83120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.441+0000 7ff91f400640 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e7b9e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff91002ee80 tx=0x7ff91000b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e0c120a1ccc11e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.441+0000 7ff90be00640 1 -- 172.21.15.6:0/3786135071 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91001c790 con 0x561323e72d80 2024-02-26T06:28:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.441+0000 7ff90be00640 1 -- 172.21.15.6:0/3786135071 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff910033070 con 0x561323e72d80 2024-02-26T06:28:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.441+0000 7ff90be00640 1 -- 172.21.15.6:0/3786135071 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910005410 con 0x561323e72d80 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.441+0000 7ff92977a740 1 -- 172.21.15.6:0/3786135071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 msgr2=0x561323e7b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.441+0000 7ff92977a740 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e7b9e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff91002ee80 tx=0x7ff91000b650 comp rx=0 tx=0).stop 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 -- 172.21.15.6:0/3786135071 shutdown_connections 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e83120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e73af0 0x561323e79b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 --2- 172.21.15.6:0/3786135071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e72d80 0x561323e7b9e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 -- 172.21.15.6:0/3786135071 >> 172.21.15.6:0/3786135071 conn(0x561323e3a480 msgr2=0x561323e722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 -- 172.21.15.6:0/3786135071 shutdown_connections 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 -- 172.21.15.6:0/3786135071 wait complete. 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 -- start start 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.442+0000 7ff92977a740 1 Processor -- start 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e72d80 0x561323e82d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 0x561323e83290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e7f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323e7f830 con 0x561323e7c300 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323e7f9a0 con 0x561323e73af0 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561323e7fb60 con 0x561323e72d80 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff92977a740 10 client.0 _ll_register_callbacks cb 0x7ff90c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff91ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 0x561323e83290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff91ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 0x561323e83290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50876/0 (socket says 172.21.15.6:50876) 2024-02-26T06:28:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.443+0000 7ff91ea00640 1 -- 172.21.15.6:0/1900977139 learned_addr learned my addr 172.21.15.6:0/1900977139 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91f400640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e72d80 0x561323e82d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91fe00640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e7f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91ea00640 1 -- 172.21.15.6:0/1900977139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e72d80 msgr2=0x561323e82d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91ea00640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e72d80 0x561323e82d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91ea00640 1 -- 172.21.15.6:0/1900977139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 msgr2=0x561323e7f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91ea00640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e7f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91ea00640 1 -- 172.21.15.6:0/1900977139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201a66a0 con 0x561323e73af0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff91ea00640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 0x561323e83290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff90c036040 tx=0x7ff90c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4322e031c0cf6e45 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff909600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff909600640 1 -- 172.21.15.6:0/1900977139 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c004100 con 0x561323e73af0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561323dfed00 con 0x561323e73af0 2024-02-26T06:28:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.444+0000 7ff909600640 1 -- 172.21.15.6:0/1900977139 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7ff90c004260 con 0x561323e73af0 2024-02-26T06:28:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff909600640 1 -- 172.21.15.6:0/1900977139 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90c004420 con 0x561323e73af0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 10 client.25553 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561323dd7c20 con 0x561323e73af0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff909600640 1 -- 172.21.15.6:0/1900977139 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561323dfed00 con 0x561323e73af0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff909600640 1 -- 172.21.15.6:0/1900977139 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff90c04a040 con 0x561323e73af0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 10 client.25553 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 20 client.25553 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323ee37a0 0x561323ee5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 4 client.25553 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.445+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561323ad9600 con 0x561323ee37a0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.446+0000 7ff91f400640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323ee37a0 0x561323ee5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.446+0000 7ff91f400640 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323ee37a0 0x561323ee5bc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bacde41d5b7a83 server_cookie=e3c977057027b23b in_seq=0 out_seq=0 2024-02-26T06:28:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.446+0000 7ff909600640 10 client.25553 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff909600640 1 -- 172.21.15.6:0/1900977139 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561323ad9600 con 0x561323ee37a0 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff909600640 10 client.25553 handle_command_reply: tid=0 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 client.25553 shutdown 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 msgr2=0x561323e83290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 0x561323e83290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff90c036040 tx=0x7ff90c009260 comp rx=0 tx=0).stop 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 shutdown_connections 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561323ee37a0 0x561323ee5bc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561323e7c300 0x561323e7f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561323e73af0 0x561323e83290 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff909600640 0 client.25553 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 --2- 172.21.15.6:0/1900977139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561323e72d80 0x561323e82d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 >> 172.21.15.6:0/1900977139 conn(0x561323e3a480 msgr2=0x561323e72050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 shutdown_connections 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.458+0000 7ff92977a740 1 -- 172.21.15.6:0/1900977139 wait complete. 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.459+0000 7ff92977a740 20 client.25553 trim_cache size 0 max 16384 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:28:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:28:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:28:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:28:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.459+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff9040a22b0 msgr2=0x7ff9040a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.459+0000 7ff926800640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff9040a22b0 0x7ff9040a4770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005e70 tx=0x7ff90c005e00 comp rx=0 tx=0).stop 2024-02-26T06:28:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.459+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920106150 msgr2=0x7ff920100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.459+0000 7ff926800640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920106150 0x7ff920100e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff91800da10 tx=0x7ff91800ded0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 shutdown_connections 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff9040a22b0 0x7ff9040a4770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff920106150 0x7ff920100e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff92006a150 0x7ff9201008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 --2- 172.21.15.6:0/3640558969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff920069790 0x7ff9201003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 >> 172.21.15.6:0/3640558969 conn(0x7ff920076e40 msgr2=0x7ff9200fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 shutdown_connections 2024-02-26T06:28:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:22.460+0000 7ff926800640 1 -- 172.21.15.6:0/3640558969 wait complete. 2024-02-26T06:28:24.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:23 smithi090 ceph-mon[35681]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-02-26T06:28:24.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:23 smithi006 ceph-mon[31790]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-02-26T06:28:24.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:23 smithi167 ceph-mon[36229]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-02-26T06:28:26.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:26.014 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:26.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:25 smithi006 ceph-mon[31790]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-02-26T06:28:26.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:25 smithi090 ceph-mon[35681]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-02-26T06:28:26.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:25 smithi167 ceph-mon[36229]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-02-26T06:28:26.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 -- 172.21.15.6:0/3362958246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0101ba0 msgr2=0x7effa0105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 --2- 172.21.15.6:0/3362958246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0101ba0 0x7effa0105be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009930 tx=0x7eff8c02bc20 comp rx=0 tx=0).stop 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 -- 172.21.15.6:0/3362958246 shutdown_connections 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 --2- 172.21.15.6:0/3362958246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa01062c0 0x7effa010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 --2- 172.21.15.6:0/3362958246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0101ba0 0x7effa0105be0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 --2- 172.21.15.6:0/3362958246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa01011f0 0x7effa01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 -- 172.21.15.6:0/3362958246 >> 172.21.15.6:0/3362958246 conn(0x7effa00fca90 msgr2=0x7effa00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 -- 172.21.15.6:0/3362958246 shutdown_connections 2024-02-26T06:28:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.258+0000 7effa5400640 1 -- 172.21.15.6:0/3362958246 wait complete. 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.259+0000 7effa5400640 1 Processor -- start 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 -- start start 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa01011f0 0x7effa019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa01062c0 0x7effa019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 0x7effa0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa019a150 con 0x7effa0199680 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa019a2c0 con 0x7effa01062c0 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa5400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa019a480 con 0x7effa01011f0 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa4a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 0x7effa0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa4a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 0x7effa0199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55534/0 (socket says 172.21.15.6:55534) 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7effa4a00640 1 -- 172.21.15.6:0/3143147010 learned_addr learned my addr 172.21.15.6:0/3143147010 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7eff9fe00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa01011f0 0x7effa019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.260+0000 7eff9f400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa01062c0 0x7effa019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa4a00640 1 -- 172.21.15.6:0/3143147010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa01011f0 msgr2=0x7effa019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa4a00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa01011f0 0x7effa019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa4a00640 1 -- 172.21.15.6:0/3143147010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa01062c0 msgr2=0x7effa019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa4a00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa01062c0 0x7effa019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa4a00640 1 -- 172.21.15.6:0/3143147010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009840 con 0x7effa0199680 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa4a00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 0x7effa0199b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7eff9800efd0 tx=0x7eff9800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7eff9cc00640 1 -- 172.21.15.6:0/3143147010 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98009b40 con 0x7effa0199680 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7eff9cc00640 1 -- 172.21.15.6:0/3143147010 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff98003b70 con 0x7effa0199680 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.261+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c009560 con 0x7effa0199680 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.262+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa01a6770 con 0x7effa0199680 2024-02-26T06:28:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.262+0000 7eff9cc00640 1 -- 172.21.15.6:0/3143147010 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980108f0 con 0x7effa0199680 2024-02-26T06:28:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.265+0000 7eff9cc00640 1 -- 172.21.15.6:0/3143147010 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7eff98003ce0 con 0x7effa0199680 2024-02-26T06:28:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.265+0000 7effa99a9740 1 Processor -- start 2024-02-26T06:28:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.265+0000 7effa99a9740 1 -- start start 2024-02-26T06:28:26.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7effa99a9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f66efe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7effa99a9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7effa99a9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7effa99a9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f665f120 con 0x5596f66f97d0 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7effa99a9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f665c650 con 0x5596f66f1140 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7effa99a9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6657e30 con 0x5596f66f0460 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7eff9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f66efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7eff9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f66efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50890/0 (socket says 172.21.15.6:50890) 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7eff9f400640 1 -- 172.21.15.6:0/401960865 learned_addr learned my addr 172.21.15.6:0/401960865 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.266+0000 7eff9f400640 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9cc00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7eff840a2340 0x7eff840a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9fe00640 1 -- 172.21.15.6:0/401960865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 msgr2=0x5596f66efe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9fe00640 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f66efe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9cc00640 1 -- 172.21.15.6:0/3143147010 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7eff98014070 con 0x7effa0199680 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9fe00640 1 -- 172.21.15.6:0/401960865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 msgr2=0x5596f66fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9fe00640 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9fe00640 1 -- 172.21.15.6:0/401960865 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f6611030 con 0x5596f66f0460 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7effa4a00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7eff840a2340 0x7eff840a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff9fe00640 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f08e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff90009c20 tx=0x7eff90011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a7f7f1246e24437 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.267+0000 7eff95200640 1 -- 172.21.15.6:0/401960865 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90004050 con 0x5596f66f0460 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7eff95200640 1 -- 172.21.15.6:0/401960865 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff90005500 con 0x5596f66f0460 2024-02-26T06:28:26.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7eff95200640 1 -- 172.21.15.6:0/401960865 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90004c30 con 0x5596f66f0460 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 -- 172.21.15.6:0/401960865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 msgr2=0x5596f66f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f08e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff90009c20 tx=0x7eff90011810 comp rx=0 tx=0).stop 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa4a00640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7eff840a2340 0x7eff840a4800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff980cbdc0 tx=0x7eff98005b50 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 -- 172.21.15.6:0/401960865 shutdown_connections 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f08e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 --2- 172.21.15.6:0/401960865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f66efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.268+0000 7effa99a9740 1 -- 172.21.15.6:0/401960865 >> 172.21.15.6:0/401960865 conn(0x5596f66b7570 msgr2=0x5596f66eaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.269+0000 7effa99a9740 1 -- 172.21.15.6:0/401960865 shutdown_connections 2024-02-26T06:28:26.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.269+0000 7effa99a9740 1 -- 172.21.15.6:0/401960865 wait complete. 2024-02-26T06:28:26.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.269+0000 7effa99a9740 1 -- start start 2024-02-26T06:28:26.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.269+0000 7effa99a9740 1 Processor -- start 2024-02-26T06:28:26.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 0x5596f66f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f97d0 0x5596f66f7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f65fce90 con 0x5596f66f0460 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f65fcb30 con 0x5596f66f97d0 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f63c61c0 con 0x5596f66f1140 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7effa99a9740 10 client.0 _ll_register_callbacks cb 0x5596f6610930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.270+0000 7eff9f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 0x5596f66f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9f400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 0x5596f66f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55548/0 (socket says 172.21.15.6:55548) 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:38646/0 (socket says 172.21.15.6:38646) 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9f400640 1 -- 172.21.15.6:0/1715733708 learned_addr learned my addr 172.21.15.6:0/1715733708 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9f400640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f97d0 0x5596f66f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 -- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f97d0 msgr2=0x5596f66f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f97d0 0x5596f66f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 -- 172.21.15.6:0/1715733708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 msgr2=0x5596f66f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 0x5596f66f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9fe00640 1 -- 172.21.15.6:0/1715733708 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff90009850 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.271+0000 7eff9f400640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 0x5596f66f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.272+0000 7eff9fe00640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66f6cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7eff90004910 tx=0x7eff90005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a30bb93824af8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.272+0000 7eff89600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.272+0000 7eff89600640 1 -- 172.21.15.6:0/1715733708 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90003a00 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.272+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596f63577e0 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.272+0000 7eff89600640 1 -- 172.21.15.6:0/1715733708 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7eff90003b60 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7eff89600640 1 -- 172.21.15.6:0/1715733708 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff90004510 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 10 client.25546 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5596f66564b0 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7eff89600640 1 -- 172.21.15.6:0/1715733708 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5596f63577e0 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7eff89600640 1 -- 172.21.15.6:0/1715733708 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff9002b020 con 0x5596f66f1140 2024-02-26T06:28:26.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 10 client.25546 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:26.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:26.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 20 client.25546 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:26.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f67097b0 0x5596f670bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 4 client.25546 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:26.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.273+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596f6394720 con 0x5596f67097b0 2024-02-26T06:28:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.274+0000 7eff9f400640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f67097b0 0x5596f670bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.274+0000 7eff9f400640 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f67097b0 0x5596f670bbd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8916ad553b42f1bb server_cookie=a6606ccb6f8e4d2d in_seq=0 out_seq=0 2024-02-26T06:28:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.274+0000 7eff89600640 10 client.25546 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7eff89600640 1 -- 172.21.15.6:0/1715733708 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596f6394720 con 0x5596f67097b0 2024-02-26T06:28:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7eff89600640 10 client.25546 handle_command_reply: tid=0 2024-02-26T06:28:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 client.25546 shutdown 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 msgr2=0x5596f66f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66f6cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7eff90004910 tx=0x7eff90005670 comp rx=0 tx=0).stop 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 shutdown_connections 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f67097b0 0x5596f670bbd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f97d0 0x5596f66f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66f6cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7eff89600640 0 client.25546 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 --2- 172.21.15.6:0/1715733708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f0460 0x5596f66f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.275+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 >> 172.21.15.6:0/1715733708 conn(0x5596f66b7570 msgr2=0x5596f66ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.276+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 shutdown_connections 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.276+0000 7effa99a9740 1 -- 172.21.15.6:0/1715733708 wait complete. 2024-02-26T06:28:26.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.276+0000 7effa99a9740 20 client.25546 trim_cache size 0 max 16384 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 Processor -- start 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 -- start start 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f6700730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f6700c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f66ebcb0 con 0x5596f66f97d0 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6705dc0 con 0x5596f66f1140 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.292+0000 7effa99a9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6706430 con 0x5596f66f0460 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f6700730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f6700730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50918/0 (socket says 172.21.15.6:50918) 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 -- 172.21.15.6:0/1504499396 learned_addr learned my addr 172.21.15.6:0/1504499396 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9f400640 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f6700c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 -- 172.21.15.6:0/1504499396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 msgr2=0x5596f66f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 -- 172.21.15.6:0/1504499396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 msgr2=0x5596f6700c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f6700c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 -- 172.21.15.6:0/1504499396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f65f2920 con 0x5596f66f1140 2024-02-26T06:28:26.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9f400640 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f6700c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:26.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff9fe00640 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f6700730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7eff90009d90 tx=0x7eff90011d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8547a3292f35791b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff8be00640 1 -- 172.21.15.6:0/1504499396 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900045f0 con 0x5596f66f1140 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff8be00640 1 -- 172.21.15.6:0/1504499396 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff900056e0 con 0x5596f66f1140 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.293+0000 7eff8be00640 1 -- 172.21.15.6:0/1504499396 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90003a60 con 0x5596f66f1140 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 -- 172.21.15.6:0/1504499396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 msgr2=0x5596f6700730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f6700730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7eff90009d90 tx=0x7eff90011d50 comp rx=0 tx=0).stop 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 -- 172.21.15.6:0/1504499396 shutdown_connections 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f6700c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f1140 0x5596f6700730 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 --2- 172.21.15.6:0/1504499396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f0460 0x5596f66f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 -- 172.21.15.6:0/1504499396 >> 172.21.15.6:0/1504499396 conn(0x5596f66b7570 msgr2=0x5596f66ea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 -- 172.21.15.6:0/1504499396 shutdown_connections 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.294+0000 7effa99a9740 1 -- 172.21.15.6:0/1504499396 wait complete. 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.295+0000 7effa99a9740 1 -- start start 2024-02-26T06:28:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.295+0000 7effa99a9740 1 Processor -- start 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 0x5596f6700a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f66fd9d0 con 0x5596f66f97d0 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f66fdb40 con 0x5596f66f0460 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f675fcf0 con 0x5596f66f1140 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7effa99a9740 10 client.0 _ll_register_callbacks cb 0x7eff8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 0x5596f6700a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 0x5596f6700a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50930/0 (socket says 172.21.15.6:50930) 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 -- 172.21.15.6:0/742175164 learned_addr learned my addr 172.21.15.6:0/742175164 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9fe00640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 -- 172.21.15.6:0/742175164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 msgr2=0x5596f66fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 -- 172.21.15.6:0/742175164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 msgr2=0x5596f66fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9f400640 1 -- 172.21.15.6:0/742175164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596f66564b0 con 0x5596f66f0460 2024-02-26T06:28:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.296+0000 7eff9fe00640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7eff9f400640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 0x5596f6700a40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7eff8c02c770 tx=0x7eff8c045040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a7fb4666262637 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7eff89600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7eff89600640 1 -- 172.21.15.6:0/742175164 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c02dea0 con 0x5596f66f0460 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7eff89600640 1 -- 172.21.15.6:0/742175164 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7eff8c004410 con 0x5596f66f0460 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff90005500 con 0x5596f66f0460 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7eff89600640 1 -- 172.21.15.6:0/742175164 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff8c04f330 con 0x5596f66f0460 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7effa99a9740 10 client.25571 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.297+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5596f6618400 con 0x5596f66f0460 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7eff89600640 1 -- 172.21.15.6:0/742175164 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff8c05f080 con 0x5596f66f0460 2024-02-26T06:28:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7eff89600640 1 -- 172.21.15.6:0/742175164 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff8c03c040 con 0x5596f66f0460 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7effa99a9740 10 client.25571 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7effa99a9740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7effa99a9740 20 client.25571 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7effa99a9740 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f6761390 0x5596f67637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7effa99a9740 4 client.25571 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5596f63e3890 con 0x5596f6761390 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.298+0000 7eff9fe00640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f6761390 0x5596f67637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7eff9fe00640 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f6761390 0x5596f67637b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88bff41fc1a6029 server_cookie=751aef1b2225af2b in_seq=0 out_seq=0 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7eff89600640 10 client.25571 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7eff89600640 1 -- 172.21.15.6:0/742175164 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5596f63e3890 con 0x5596f6761390 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7eff89600640 10 client.25571 handle_command_reply: tid=0 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 client.25571 shutdown 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 msgr2=0x5596f6700a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 0x5596f6700a40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7eff8c02c770 tx=0x7eff8c045040 comp rx=0 tx=0).stop 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 shutdown_connections 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5596f6761390 0x5596f67637b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f66f97d0 0x5596f66fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596f66f1140 0x5596f66fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 --2- 172.21.15.6:0/742175164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596f66f0460 0x5596f6700a40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7eff89600640 0 client.25571 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 >> 172.21.15.6:0/742175164 conn(0x5596f66b7570 msgr2=0x5596f66f4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 shutdown_connections 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 1 -- 172.21.15.6:0/742175164 wait complete. 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.299+0000 7effa99a9740 20 client.25571 trim_cache size 0 max 16384 2024-02-26T06:28:26.302 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7eff840a2340 msgr2=0x7eff840a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7eff840a2340 0x7eff840a4800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff980cbdc0 tx=0x7eff98005b50 comp rx=0 tx=0).stop 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 msgr2=0x7effa0199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 0x7effa0199b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7eff9800efd0 tx=0x7eff9800c6a0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 shutdown_connections 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7eff840a2340 0x7eff840a4800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effa0199680 0x7effa0199b00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa01062c0 0x7effa019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 --2- 172.21.15.6:0/3143147010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa01011f0 0x7effa019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 >> 172.21.15.6:0/3143147010 conn(0x7effa00fca90 msgr2=0x7effa00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 shutdown_connections 2024-02-26T06:28:26.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.300+0000 7effa5400640 1 -- 172.21.15.6:0/3143147010 wait complete. 2024-02-26T06:28:26.316 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:28:26.316 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:28:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.523+0000 7f832e800640 1 -- 172.21.15.6:0/3955396008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 msgr2=0x7f832810b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.523+0000 7f832e800640 1 --2- 172.21.15.6:0/3955396008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 0x7f832810b080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8314009900 tx=0x7f831402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 -- 172.21.15.6:0/3955396008 shutdown_connections 2024-02-26T06:28:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 --2- 172.21.15.6:0/3955396008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832810b650 0x7f832810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 --2- 172.21.15.6:0/3955396008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 0x7f832810b080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 --2- 172.21.15.6:0/3955396008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f8328102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 -- 172.21.15.6:0/3955396008 >> 172.21.15.6:0/3955396008 conn(0x7f83280fb9e0 msgr2=0x7f83280fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 -- 172.21.15.6:0/3955396008 shutdown_connections 2024-02-26T06:28:26.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.524+0000 7f832e800640 1 -- 172.21.15.6:0/3955396008 wait complete. 2024-02-26T06:28:26.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 Processor -- start 2024-02-26T06:28:26.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 -- start start 2024-02-26T06:28:26.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f832819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 0x7f832819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832810b650 0x7f83281996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328199e80 con 0x7f8328102280 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328199ff0 con 0x7f8328102ba0 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832819a1b0 con 0x7f832810b650 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.525+0000 7f832d400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f832819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f832819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55582/0 (socket says 172.21.15.6:55582) 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 -- 172.21.15.6:0/3698245338 learned_addr learned my addr 172.21.15.6:0/3698245338 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832ca00640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 0x7f832819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832de00640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832810b650 0x7f83281996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 -- 172.21.15.6:0/3698245338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832810b650 msgr2=0x7f83281996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832810b650 0x7f83281996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 -- 172.21.15.6:0/3698245338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 msgr2=0x7f832819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 0x7f832819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 -- 172.21.15.6:0/3698245338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8314009560 con 0x7f8328102280 2024-02-26T06:28:26.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.526+0000 7f832d400640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f832819f630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e9f0 tx=0x7f831c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.527+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c009700 con 0x7f8328102280 2024-02-26T06:28:26.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.527+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f831c00cd80 con 0x7f8328102280 2024-02-26T06:28:26.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.527+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832819a490 con 0x7f8328102280 2024-02-26T06:28:26.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.527+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c018920 con 0x7f8328102280 2024-02-26T06:28:26.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.527+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83281a65b0 con 0x7f8328102280 2024-02-26T06:28:26.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.528+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8328068c90 con 0x7f8328102280 2024-02-26T06:28:26.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.532+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f831c018a80 con 0x7f8328102280 2024-02-26T06:28:26.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.533+0000 7f8326000640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f83100a22b0 0x7f83100a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:26.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.533+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f831c014070 con 0x7f8328102280 2024-02-26T06:28:26.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.533+0000 7f832ca00640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f83100a22b0 0x7f83100a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:26.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.534+0000 7f832ca00640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f83100a22b0 0x7f83100a4770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8314005ab0 tx=0x7f8314005a20 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:26.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.542+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f831c052d20 con 0x7f8328102280 2024-02-26T06:28:26.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.779+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832804ef50 con 0x7f8328102280 2024-02-26T06:28:26.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.780+0000 7f8326000640 1 -- 172.21.15.6:0/3698245338 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f831c097170 con 0x7f8328102280 2024-02-26T06:28:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:28:26.780 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:27:45.824981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:28:26.780 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f83100a22b0 msgr2=0x7f83100a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f83100a22b0 0x7f83100a4770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8314005ab0 tx=0x7f8314005a20 comp rx=0 tx=0).stop 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 msgr2=0x7f832819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f832819f630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e9f0 tx=0x7f831c00eeb0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 shutdown_connections 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f83100a22b0 0x7f83100a4770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832810b650 0x7f83281996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8328102ba0 0x7f832819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 --2- 172.21.15.6:0/3698245338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8328102280 0x7f832819f630 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:26.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 >> 172.21.15.6:0/3698245338 conn(0x7f83280fb9e0 msgr2=0x7f83280fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:26.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 shutdown_connections 2024-02-26T06:28:26.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:26.783+0000 7f832e800640 1 -- 172.21.15.6:0/3698245338 wait complete. 2024-02-26T06:28:26.799 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:26.799 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-26T06:28:27.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 -- 172.21.15.6:0/1002280855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 msgr2=0x7f7e0c101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 --2- 172.21.15.6:0/1002280855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c101760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009cd0 tx=0x7f7dfc02c0e0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 -- 172.21.15.6:0/1002280855 shutdown_connections 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 --2- 172.21.15.6:0/1002280855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 0x7f7e0c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 --2- 172.21.15.6:0/1002280855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c101d30 0x7f7e0c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 --2- 172.21.15.6:0/1002280855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c101760 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 -- 172.21.15.6:0/1002280855 >> 172.21.15.6:0/1002280855 conn(0x7f7e0c0fcc20 msgr2=0x7f7e0c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 -- 172.21.15.6:0/1002280855 shutdown_connections 2024-02-26T06:28:27.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.008+0000 7f7e13600640 1 -- 172.21.15.6:0/1002280855 wait complete. 2024-02-26T06:28:27.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 Processor -- start 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 -- start start 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 0x7f7e0c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c199910 0x7f7e0c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c19a300 con 0x7f7e0c106450 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c19a470 con 0x7f7e0c101380 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.009+0000 7f7e13600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c19a630 con 0x7f7e0c199910 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48338/0 (socket says 172.21.15.6:48338) 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 -- 172.21.15.6:0/510990289 learned_addr learned my addr 172.21.15.6:0/510990289 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e03e00640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 0x7f7e0c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e11600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c199910 0x7f7e0c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 -- 172.21.15.6:0/510990289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c199910 msgr2=0x7f7e0c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c199910 0x7f7e0c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 -- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 msgr2=0x7f7e0c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 0x7f7e0c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e10c00640 1 -- 172.21.15.6:0/510990289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc009900 con 0x7f7e0c101380 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.010+0000 7f7e03e00640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 0x7f7e0c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.011+0000 7f7e10c00640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c19f710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc004af0 tx=0x7f7dfc034cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.011+0000 7f7e0a000640 1 -- 172.21.15.6:0/510990289 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0328c0 con 0x7f7e0c101380 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.011+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c1a65d0 con 0x7f7e0c101380 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.011+0000 7f7e0a000640 1 -- 172.21.15.6:0/510990289 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7dfc003e50 con 0x7f7e0c101380 2024-02-26T06:28:27.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.011+0000 7f7e0a000640 1 -- 172.21.15.6:0/510990289 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc040450 con 0x7f7e0c101380 2024-02-26T06:28:27.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.011+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c1a6a80 con 0x7f7e0c101380 2024-02-26T06:28:27.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.014+0000 7f7e0a000640 1 -- 172.21.15.6:0/510990289 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f7dfc00a040 con 0x7f7e0c101380 2024-02-26T06:28:27.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.014+0000 7f7e16636740 1 Processor -- start 2024-02-26T06:28:27.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.014+0000 7f7e16636740 1 -- start start 2024-02-26T06:28:27.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e16636740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b432ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e16636740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b433530 0x55744b43bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e16636740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 0x55744b43e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e16636740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b3a11e0 con 0x55744b43c110 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e16636740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b39e710 con 0x55744b433530 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e16636740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b399ef0 con 0x55744b432c10 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 0x55744b43e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 0x55744b43e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53868/0 (socket says 172.21.15.6:53868) 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 -- 172.21.15.6:0/3947988006 learned_addr learned my addr 172.21.15.6:0/3947988006 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e10c00640 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b433530 0x55744b43bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e11600640 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b432ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e0a000640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7df00a22c0 0x7f7df00a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 -- 172.21.15.6:0/3947988006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 msgr2=0x55744b432ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b432ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 -- 172.21.15.6:0/3947988006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b433530 msgr2=0x55744b43bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.015+0000 7f7e03e00640 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b433530 0x55744b43bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e03e00640 1 -- 172.21.15.6:0/3947988006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744b35bdc0 con 0x55744b43c110 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e11600640 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b432ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e0a000640 1 -- 172.21.15.6:0/510990289 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7dfc0f0ef0 con 0x7f7e0c101380 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e11600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7df00a22c0 0x7f7df00a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e03e00640 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 0x55744b43e500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7df4010400 tx=0x7f7df40108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f07f1a688fa551 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e01c00640 1 -- 172.21.15.6:0/3947988006 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4011760 con 0x55744b43c110 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e01c00640 1 -- 172.21.15.6:0/3947988006 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df4015070 con 0x55744b43c110 2024-02-26T06:28:27.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.016+0000 7f7e01c00640 1 -- 172.21.15.6:0/3947988006 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df401cd30 con 0x55744b43c110 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 -- 172.21.15.6:0/3947988006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 msgr2=0x55744b43e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e11600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7df00a22c0 0x7f7df00a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009ef0 tx=0x7f7e040046c0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 0x55744b43e500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7df4010400 tx=0x7f7df40108c0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 -- 172.21.15.6:0/3947988006 shutdown_connections 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b43c110 0x55744b43e500 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b433530 0x55744b43bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 --2- 172.21.15.6:0/3947988006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b432ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 -- 172.21.15.6:0/3947988006 >> 172.21.15.6:0/3947988006 conn(0x55744b42d340 msgr2=0x55744b42d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 -- 172.21.15.6:0/3947988006 shutdown_connections 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.017+0000 7f7e16636740 1 -- 172.21.15.6:0/3947988006 wait complete. 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.018+0000 7f7e16636740 1 -- start start 2024-02-26T06:28:27.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.018+0000 7f7e16636740 1 Processor -- start 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b439080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b33f120 con 0x55744b433530 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b33edc0 con 0x55744b43c110 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b1081c0 con 0x55744b432c10 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e16636740 10 client.0 _ll_register_callbacks cb 0x55744b38e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53878/0 (socket says 172.21.15.6:53878) 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 -- 172.21.15.6:0/538834144 learned_addr learned my addr 172.21.15.6:0/538834144 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e10c00640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b439080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e11600640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 -- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 msgr2=0x55744b439080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b439080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 -- 172.21.15.6:0/538834144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 msgr2=0x55744b439b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.019+0000 7f7e03e00640 1 -- 172.21.15.6:0/538834144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df400bc60 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7e03e00640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4395c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7df401c990 tx=0x7f7df401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39013a681b69698 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7dfa000640 1 -- 172.21.15.6:0/538834144 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4021070 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7dfa000640 1 -- 172.21.15.6:0/538834144 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7df4017070 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55744b0997e0 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7dfa000640 1 -- 172.21.15.6:0/538834144 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7df401c480 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7e16636740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.020+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55744b398570 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7dfa000640 1 -- 172.21.15.6:0/538834144 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df4032020 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7dfa000640 1 -- 172.21.15.6:0/538834144 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7df4004750 con 0x55744b433530 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e16636740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e16636740 10 client.15450 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-02-26T06:28:27.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e16636740 20 client.15450 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e16636740 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b44bf20 0x55744b44e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e16636740 4 client.15450 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:27.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55744b0d6720 con 0x55744b44bf20 2024-02-26T06:28:27.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.021+0000 7f7e10c00640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b44bf20 0x55744b44e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.022+0000 7f7e10c00640 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b44bf20 0x55744b44e360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72a8cd6d85ec281 server_cookie=4429fb892fdb1d1e in_seq=0 out_seq=0 2024-02-26T06:28:27.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.022+0000 7f7dfa000640 10 client.15450 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:28:27.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7dfa000640 1 -- 172.21.15.6:0/538834144 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55744b0d6720 con 0x55744b44bf20 2024-02-26T06:28:27.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7dfa000640 10 client.15450 handle_command_reply: tid=0 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 client.15450 shutdown 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 msgr2=0x55744b4395c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4395c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7df401c990 tx=0x7f7df401cc60 comp rx=0 tx=0).stop 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 shutdown_connections 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b44bf20 0x55744b44e360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4395c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 --2- 172.21.15.6:0/538834144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b432c10 0x55744b439080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7dfa000640 0 client.15450 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 >> 172.21.15.6:0/538834144 conn(0x55744b42d340 msgr2=0x55744b42f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.023+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 shutdown_connections 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.024+0000 7f7e16636740 1 -- 172.21.15.6:0/538834144 wait complete. 2024-02-26T06:28:27.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.024+0000 7f7e16636740 20 client.15450 trim_cache size 0 max 16384 2024-02-26T06:28:27.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 Processor -- start 2024-02-26T06:28:27.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 -- start start 2024-02-26T06:28:27.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b432c10 0x55744b43b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 0x55744b42d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b447ed0 con 0x55744b432c10 2024-02-26T06:28:27.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b4535b0 con 0x55744b43c110 2024-02-26T06:28:27.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.040+0000 7f7e16636740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b4375d0 con 0x55744b433530 2024-02-26T06:28:27.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 0x55744b42d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 0x55744b42d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58478/0 (socket says 172.21.15.6:58478) 2024-02-26T06:28:27.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 -- 172.21.15.6:0/1449865830 learned_addr learned my addr 172.21.15.6:0/1449865830 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e10c00640 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b432c10 0x55744b43b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e11600640 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 -- 172.21.15.6:0/1449865830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 msgr2=0x55744b439920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 -- 172.21.15.6:0/1449865830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b432c10 msgr2=0x55744b43b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b432c10 0x55744b43b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 -- 172.21.15.6:0/1449865830 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744b334a80 con 0x55744b433530 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e11600640 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.041+0000 7f7e03e00640 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 0x55744b42d720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7df400b3d0 tx=0x7f7df400bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f91dd4cbc687b8a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.042+0000 7f7e01200640 1 -- 172.21.15.6:0/1449865830 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4021040 con 0x55744b433530 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.042+0000 7f7e01200640 1 -- 172.21.15.6:0/1449865830 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df4032070 con 0x55744b433530 2024-02-26T06:28:27.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.042+0000 7f7e01200640 1 -- 172.21.15.6:0/1449865830 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df40042a0 con 0x55744b433530 2024-02-26T06:28:27.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 -- 172.21.15.6:0/1449865830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 msgr2=0x55744b42d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 0x55744b42d720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7df400b3d0 tx=0x7f7df400bf00 comp rx=0 tx=0).stop 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 -- 172.21.15.6:0/1449865830 shutdown_connections 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b43c110 0x55744b439920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b433530 0x55744b42d720 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 --2- 172.21.15.6:0/1449865830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b432c10 0x55744b43b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 -- 172.21.15.6:0/1449865830 >> 172.21.15.6:0/1449865830 conn(0x55744b42d340 msgr2=0x55744b433910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 -- 172.21.15.6:0/1449865830 shutdown_connections 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 -- 172.21.15.6:0/1449865830 wait complete. 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.043+0000 7f7e16636740 1 -- start start 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 Processor -- start 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 0x55744b443220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b43c110 0x55744b441b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b442090 con 0x55744b433530 2024-02-26T06:28:27.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b4421d0 con 0x55744b432c10 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744b442390 con 0x55744b43c110 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.044+0000 7f7e16636740 10 client.0 _ll_register_callbacks cb 0x7f7dfc0ee4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 0x55744b443220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 0x55744b443220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48358/0 (socket says 172.21.15.6:48358) 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 -- 172.21.15.6:0/703262290 learned_addr learned my addr 172.21.15.6:0/703262290 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e11600640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b43c110 0x55744b441b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e03e00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 -- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b43c110 msgr2=0x55744b441b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b43c110 0x55744b441b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 -- 172.21.15.6:0/703262290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 msgr2=0x55744b4415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 -- 172.21.15.6:0/703262290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55744b398570 con 0x55744b432c10 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e03e00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:27.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e11600640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b43c110 0x55744b441b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7e10c00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 0x55744b443220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0bd850 tx=0x7f7dfc02c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef1894f0b16c778 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.045+0000 7f7dfa000640 1 -- 172.21.15.6:0/703262290 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0bd400 con 0x55744b432c10 2024-02-26T06:28:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7dfa000640 1 -- 172.21.15.6:0/703262290 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7dfc032a20 con 0x55744b432c10 2024-02-26T06:28:27.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7df4015070 con 0x55744b432c10 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7dfa000640 1 -- 172.21.15.6:0/703262290 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dfc0bd220 con 0x55744b432c10 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 10 client.25595 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55744b35a620 con 0x55744b432c10 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7dfa000640 1 -- 172.21.15.6:0/703262290 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7dfc0da070 con 0x55744b432c10 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7dfa000640 1 -- 172.21.15.6:0/703262290 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7dfc004040 con 0x55744b432c10 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 10 client.25595 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 10 client.25595 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 20 client.25595 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b4a3830 0x55744b4a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 4 client.25595 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:27.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.046+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55744b125890 con 0x55744b4a3830 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.047+0000 7f7e03e00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b4a3830 0x55744b4a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.047+0000 7f7e03e00640 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b4a3830 0x55744b4a5c50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3448039bf6e3860 server_cookie=e075e9863898cb1c in_seq=0 out_seq=0 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.047+0000 7f7dfa000640 10 client.25595 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.047+0000 7f7dfa000640 1 -- 172.21.15.6:0/703262290 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55744b125890 con 0x55744b4a3830 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.047+0000 7f7dfa000640 10 client.25595 handle_command_reply: tid=0 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.047+0000 7f7e16636740 1 client.25595 shutdown 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 msgr2=0x55744b443220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 0x55744b443220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0bd850 tx=0x7f7dfc02c0e0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 shutdown_connections 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55744b4a3830 0x55744b4a5c50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55744b43c110 0x55744b441b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55744b433530 0x55744b4415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 --2- 172.21.15.6:0/703262290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55744b432c10 0x55744b443220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 >> 172.21.15.6:0/703262290 conn(0x55744b42d340 msgr2=0x55744b42d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7dfa000640 0 client.25595 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:28:27.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 shutdown_connections 2024-02-26T06:28:27.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 1 -- 172.21.15.6:0/703262290 wait complete. 2024-02-26T06:28:27.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.048+0000 7f7e16636740 20 client.25595 trim_cache size 0 max 16384 2024-02-26T06:28:27.051 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7df00a22c0 msgr2=0x7f7df00a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7df00a22c0 0x7f7df00a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009ef0 tx=0x7f7e040046c0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 msgr2=0x7f7e0c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c19f710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc004af0 tx=0x7f7dfc034cc0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 shutdown_connections 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7df00a22c0 0x7f7df00a4780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c199910 0x7f7e0c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e0c106450 0x7f7e0c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 --2- 172.21.15.6:0/510990289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c101380 0x7f7e0c19f710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 >> 172.21.15.6:0/510990289 conn(0x7f7e0c0fcc20 msgr2=0x7f7e0c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 shutdown_connections 2024-02-26T06:28:27.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.049+0000 7f7e13600640 1 -- 172.21.15.6:0/510990289 wait complete. 2024-02-26T06:28:27.062 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:26 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3698245338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:27.065 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:27.065 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-26T06:28:27.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:26 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3698245338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:27.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:26 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3698245338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 -- 172.21.15.6:0/3862137225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb8108660 msgr2=0x7f5eb8110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 --2- 172.21.15.6:0/3862137225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb8108660 0x7f5eb8110230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009c10 tx=0x7f5eb002bef0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 -- 172.21.15.6:0/3862137225 shutdown_connections 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 --2- 172.21.15.6:0/3862137225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb8108660 0x7f5eb8110230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 --2- 172.21.15.6:0/3862137225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eb8103f40 0x7f5eb8107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 --2- 172.21.15.6:0/3862137225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8103590 0x7f5eb8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 -- 172.21.15.6:0/3862137225 >> 172.21.15.6:0/3862137225 conn(0x7f5eb80fedd0 msgr2=0x7f5eb81011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.299+0000 7f5ec0400640 1 -- 172.21.15.6:0/3862137225 shutdown_connections 2024-02-26T06:28:27.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.300+0000 7f5ec0400640 1 -- 172.21.15.6:0/3862137225 wait complete. 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 Processor -- start 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 -- start start 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eb8103590 0x7f5eb819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 0x7f5eb819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 0x7f5eb8199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819a290 con 0x7f5eb8108660 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819a400 con 0x7f5eb81997e0 2024-02-26T06:28:27.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.301+0000 7f5ec0400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819a570 con 0x7f5eb8103590 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebd000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 0x7f5eb819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 0x7f5eb8199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 0x7f5eb8199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48378/0 (socket says 172.21.15.6:48378) 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebd000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 0x7f5eb819fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53930/0 (socket says 172.21.15.6:53930) 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 -- 172.21.15.6:0/1506466907 learned_addr learned my addr 172.21.15.6:0/1506466907 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebda00640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eb8103590 0x7f5eb819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 -- 172.21.15.6:0/1506466907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eb8103590 msgr2=0x7f5eb819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eb8103590 0x7f5eb819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 -- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 msgr2=0x7f5eb819fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 0x7f5eb819fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebe400640 1 -- 172.21.15.6:0/1506466907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009840 con 0x7f5eb81997e0 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.302+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 0x7f5eb819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.303+0000 7f5ebe400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 0x7f5eb8199c90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0036040 tx=0x7f5eb002cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.303+0000 7f5eb6000640 1 -- 172.21.15.6:0/1506466907 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003970 con 0x7f5eb81997e0 2024-02-26T06:28:27.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.303+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb81a64c0 con 0x7f5eb81997e0 2024-02-26T06:28:27.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.303+0000 7f5eb6000640 1 -- 172.21.15.6:0/1506466907 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb0004040 con 0x7f5eb81997e0 2024-02-26T06:28:27.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.303+0000 7f5eb6000640 1 -- 172.21.15.6:0/1506466907 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb003d720 con 0x7f5eb81997e0 2024-02-26T06:28:27.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.303+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb81a6940 con 0x7f5eb81997e0 2024-02-26T06:28:27.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.307+0000 7f5eb6000640 1 -- 172.21.15.6:0/1506466907 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f5eb0003ad0 con 0x7f5eb81997e0 2024-02-26T06:28:27.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5ec3243740 1 Processor -- start 2024-02-26T06:28:27.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5ec3243740 1 -- start start 2024-02-26T06:28:27.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5eb6000640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea00a2340 0x7f5ea00a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5eb6000640 1 -- 172.21.15.6:0/1506466907 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5eb00f0ab0 con 0x7f5eb81997e0 2024-02-26T06:28:27.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.308+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ec3243740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc1041e0 con 0x5629cc195c10 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ec3243740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc101710 con 0x5629cc196530 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ec3243740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc0fcef0 con 0x5629cc19f110 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea00a2340 0x7f5ea00a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48392/0 (socket says 172.21.15.6:48392) 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 -- 172.21.15.6:0/3076072667 learned_addr learned my addr 172.21.15.6:0/3076072667 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebda00640 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 -- 172.21.15.6:0/3076072667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 msgr2=0x5629cc1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 -- 172.21.15.6:0/3076072667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 msgr2=0x5629cc195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebd000640 1 -- 172.21.15.6:0/3076072667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629cc0bedc0 con 0x5629cc196530 2024-02-26T06:28:27.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.309+0000 7f5ebda00640 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:27.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.310+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea00a2340 0x7f5ea00a4800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009710 tx=0x7f5ea4009e80 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.310+0000 7f5ebd000640 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19ebd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4005f60 tx=0x7f5ea401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda2f0dc325e4b8b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.310+0000 7f5eaaa00640 1 -- 172.21.15.6:0/3076072667 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ec10 con 0x5629cc196530 2024-02-26T06:28:27.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.310+0000 7f5eaaa00640 1 -- 172.21.15.6:0/3076072667 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea400ed70 con 0x5629cc196530 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.310+0000 7f5eaaa00640 1 -- 172.21.15.6:0/3076072667 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea401f5f0 con 0x5629cc196530 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 -- 172.21.15.6:0/3076072667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 msgr2=0x5629cc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19ebd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4005f60 tx=0x7f5ea401d000 comp rx=0 tx=0).stop 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 -- 172.21.15.6:0/3076072667 shutdown_connections 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19ebd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 --2- 172.21.15.6:0/3076072667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 -- 172.21.15.6:0/3076072667 >> 172.21.15.6:0/3076072667 conn(0x5629cc190340 msgr2=0x5629cc190720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.311+0000 7f5ec3243740 1 -- 172.21.15.6:0/3076072667 shutdown_connections 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.312+0000 7f5ec3243740 1 -- 172.21.15.6:0/3076072667 wait complete. 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.312+0000 7f5ec3243740 1 -- start start 2024-02-26T06:28:27.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.312+0000 7f5ec3243740 1 Processor -- start 2024-02-26T06:28:27.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc0a2120 con 0x5629cc19f110 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc0a1dc0 con 0x5629cc196530 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cbe6b1c0 con 0x5629cc195c10 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ec3243740 10 client.0 _ll_register_callbacks cb 0x5629cc0f1280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ebd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ebd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48400/0 (socket says 172.21.15.6:48400) 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ebd000640 1 -- 172.21.15.6:0/2812753301 learned_addr learned my addr 172.21.15.6:0/2812753301 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ebda00640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.313+0000 7f5ebe400640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5ebd000640 1 -- 172.21.15.6:0/2812753301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 msgr2=0x5629cc19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5ebd000640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5ebd000640 1 -- 172.21.15.6:0/2812753301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 msgr2=0x5629cc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5ebd000640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5ebd000640 1 -- 172.21.15.6:0/2812753301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea401b040 con 0x5629cc196530 2024-02-26T06:28:27.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5ebd000640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19c750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40137b0 tx=0x7f5ea4005370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd41de6369db092d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5e9f400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.314+0000 7f5e9f400640 1 -- 172.21.15.6:0/2812753301 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ec10 con 0x5629cc196530 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.315+0000 7f5e9f400640 1 -- 172.21.15.6:0/2812753301 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5ea4021410 con 0x5629cc196530 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.315+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629cbdfc7e0 con 0x5629cc196530 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.315+0000 7f5e9f400640 1 -- 172.21.15.6:0/2812753301 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea40215d0 con 0x5629cc196530 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.315+0000 7f5ec3243740 10 client.25607 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.315+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5629cc0fb570 con 0x5629cc196530 2024-02-26T06:28:27.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.315+0000 7f5e9f400640 1 -- 172.21.15.6:0/2812753301 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5629cbdfc7e0 con 0x5629cc196530 2024-02-26T06:28:27.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.316+0000 7f5e9f400640 1 -- 172.21.15.6:0/2812753301 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ea4021730 con 0x5629cc196530 2024-02-26T06:28:27.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.316+0000 7f5ec3243740 10 client.25607 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.316+0000 7f5ec3243740 10 client.25607 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-02-26T06:28:27.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.316+0000 7f5ec3243740 20 client.25607 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.316+0000 7f5ec3243740 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc1af160 0x5629cc1b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.317+0000 7f5ec3243740 4 client.25607 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:27.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.317+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629cbe39720 con 0x5629cc1af160 2024-02-26T06:28:27.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.317+0000 7f5ebda00640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc1af160 0x5629cc1b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.318+0000 7f5ebda00640 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc1af160 0x5629cc1b1580 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fa8b7ea5ce54d3 server_cookie=e5cbbd3f2ebfaf0f in_seq=0 out_seq=0 2024-02-26T06:28:27.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.318+0000 7f5e9f400640 10 client.25607 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5e9f400640 1 -- 172.21.15.6:0/2812753301 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629cbe39720 con 0x5629cc1af160 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5e9f400640 10 client.25607 handle_command_reply: tid=0 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 client.25607 shutdown 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 msgr2=0x5629cc19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19c750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40137b0 tx=0x7f5ea4005370 comp rx=0 tx=0).stop 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 shutdown_connections 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc1af160 0x5629cc1b1580 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19c750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 --2- 172.21.15.6:0/2812753301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 >> 172.21.15.6:0/2812753301 conn(0x5629cc190340 msgr2=0x5629cc192090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5e9f400640 0 client.25607 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:28:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 shutdown_connections 2024-02-26T06:28:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 1 -- 172.21.15.6:0/2812753301 wait complete. 2024-02-26T06:28:27.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.319+0000 7f5ec3243740 20 client.25607 trim_cache size 0 max 16384 2024-02-26T06:28:27.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.343+0000 7f5ec3243740 1 Processor -- start 2024-02-26T06:28:27.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.343+0000 7f5ec3243740 1 -- start start 2024-02-26T06:28:27.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc199470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ec3243740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc195650 con 0x5629cc19f110 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ec3243740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc18fbc0 con 0x5629cc196530 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ec3243740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc1b67b0 con 0x5629cc195c10 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ebe400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ebda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ebe400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53976/0 (socket says 172.21.15.6:53976) 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.344+0000 7f5ebe400640 1 -- 172.21.15.6:0/4226271486 learned_addr learned my addr 172.21.15.6:0/4226271486 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebd000640 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebda00640 1 -- 172.21.15.6:0/4226271486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 msgr2=0x5629cc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebda00640 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebda00640 1 -- 172.21.15.6:0/4226271486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 msgr2=0x5629cc199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebda00640 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebda00640 1 -- 172.21.15.6:0/4226271486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629cc097a80 con 0x5629cc195c10 2024-02-26T06:28:27.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebe400640 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.345+0000 7f5ebda00640 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19e940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5629cc1b1b60 tx=0x7f5eac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=689833f99e996627 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.346+0000 7f5eaa000640 1 -- 172.21.15.6:0/4226271486 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac017070 con 0x5629cc195c10 2024-02-26T06:28:27.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.346+0000 7f5eaa000640 1 -- 172.21.15.6:0/4226271486 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eac018040 con 0x5629cc195c10 2024-02-26T06:28:27.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.346+0000 7f5eaa000640 1 -- 172.21.15.6:0/4226271486 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0107b0 con 0x5629cc195c10 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 -- 172.21.15.6:0/4226271486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 msgr2=0x5629cc19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19e940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5629cc1b1b60 tx=0x7f5eac009260 comp rx=0 tx=0).stop 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 -- 172.21.15.6:0/4226271486 shutdown_connections 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc19f110 0x5629cc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 --2- 172.21.15.6:0/4226271486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc195c10 0x5629cc19e940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 -- 172.21.15.6:0/4226271486 >> 172.21.15.6:0/4226271486 conn(0x5629cc190340 msgr2=0x5629cc197490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 -- 172.21.15.6:0/4226271486 shutdown_connections 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.347+0000 7f5ec3243740 1 -- 172.21.15.6:0/4226271486 wait complete. 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.348+0000 7f5ec3243740 1 -- start start 2024-02-26T06:28:27.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.348+0000 7f5ec3243740 1 Processor -- start 2024-02-26T06:28:27.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc1a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc205800 con 0x5629cc195c10 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc205970 con 0x5629cc196530 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cc205b30 con 0x5629cc19f110 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ec3243740 10 client.0 _ll_register_callbacks cb 0x7f5ea4013860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ebda00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ebda00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc1a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53986/0 (socket says 172.21.15.6:53986) 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ebda00640 1 -- 172.21.15.6:0/1184855843 learned_addr learned my addr 172.21.15.6:0/1184855843 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc1a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ebe400640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.349+0000 7f5ebda00640 1 -- 172.21.15.6:0/1184855843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 msgr2=0x5629cc1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ebda00640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ebda00640 1 -- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 msgr2=0x5629cc1a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ebda00640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc1a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ebda00640 1 -- 172.21.15.6:0/1184855843 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb81a6940 con 0x5629cc195c10 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ebda00640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc1a4c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5eac005760 tx=0x7f5eac003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044db687030453c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5e9f400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5e9f400640 1 -- 172.21.15.6:0/1184855843 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac017070 con 0x5629cc195c10 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea400ed70 con 0x5629cc195c10 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.350+0000 7f5e9f400640 1 -- 172.21.15.6:0/1184855843 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5eac003da0 con 0x5629cc195c10 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5e9f400640 1 -- 172.21.15.6:0/1184855843 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eac01f450 con 0x5629cc195c10 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f5eac009a20 con 0x5629cc195c10 2024-02-26T06:28:27.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5e9f400640 1 -- 172.21.15.6:0/1184855843 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea400ed70 con 0x5629cc195c10 2024-02-26T06:28:27.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5e9f400640 1 -- 172.21.15.6:0/1184855843 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5eac026070 con 0x5629cc195c10 2024-02-26T06:28:27.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 10 client.15474 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 20 client.15474 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc207300 0x5629cc209720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 4 client.15474 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.351+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629cbe88890 con 0x5629cc207300 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.352+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc207300 0x5629cc209720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.352+0000 7f5ebd000640 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc207300 0x5629cc209720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d25052b3658d2a9b server_cookie=9677829826622f78 in_seq=0 out_seq=0 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.352+0000 7f5e9f400640 10 client.15474 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5e9f400640 1 -- 172.21.15.6:0/1184855843 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5629cbe88890 con 0x5629cc207300 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5e9f400640 10 client.15474 handle_command_reply: tid=0 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 client.15474 shutdown 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 msgr2=0x5629cc1a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc1a4c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5eac005760 tx=0x7f5eac003b80 comp rx=0 tx=0).stop 2024-02-26T06:28:27.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 shutdown_connections 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x5629cc207300 0x5629cc209720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629cc19f110 0x5629cc1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629cc196530 0x5629cc1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 --2- 172.21.15.6:0/1184855843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629cc195c10 0x5629cc1a4c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 >> 172.21.15.6:0/1184855843 conn(0x5629cc190340 msgr2=0x5629cc18ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5e9f400640 0 client.15474 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 shutdown_connections 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 1 -- 172.21.15.6:0/1184855843 wait complete. 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.353+0000 7f5ec3243740 20 client.15474 trim_cache size 0 max 16384 2024-02-26T06:28:27.355 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:28:27.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea00a2340 msgr2=0x7f5ea00a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea00a2340 0x7f5ea00a4800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009710 tx=0x7f5ea4009e80 comp rx=0 tx=0).stop 2024-02-26T06:28:27.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 msgr2=0x7f5eb8199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 0x7f5eb8199c90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0036040 tx=0x7f5eb002cb70 comp rx=0 tx=0).stop 2024-02-26T06:28:27.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 shutdown_connections 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea00a2340 0x7f5ea00a4800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5eb81997e0 0x7f5eb8199c90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5eb8108660 0x7f5eb819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 --2- 172.21.15.6:0/1506466907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eb8103590 0x7f5eb819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 >> 172.21.15.6:0/1506466907 conn(0x7f5eb80fedd0 msgr2=0x7f5eb81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 shutdown_connections 2024-02-26T06:28:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.355+0000 7f5ec0400640 1 -- 172.21.15.6:0/1506466907 wait complete. 2024-02-26T06:28:27.371 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:27.372 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 -- 172.21.15.6:0/2841577836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 msgr2=0x7f4f6c103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 --2- 172.21.15.6:0/2841577836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c103990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009be0 tx=0x7f4f5c02bef0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 -- 172.21.15.6:0/2841577836 shutdown_connections 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 --2- 172.21.15.6:0/2841577836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c108680 0x7f4f6c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 --2- 172.21.15.6:0/2841577836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c103f60 0x7f4f6c107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 --2- 172.21.15.6:0/2841577836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c103990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 -- 172.21.15.6:0/2841577836 >> 172.21.15.6:0/2841577836 conn(0x7f4f6c0fedf0 msgr2=0x7f4f6c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 -- 172.21.15.6:0/2841577836 shutdown_connections 2024-02-26T06:28:27.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.555+0000 7f4f70a00640 1 -- 172.21.15.6:0/2841577836 wait complete. 2024-02-26T06:28:27.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.556+0000 7f4f70a00640 1 Processor -- start 2024-02-26T06:28:27.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.556+0000 7f4f70a00640 1 -- start start 2024-02-26T06:28:27.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f70a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f70a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c108680 0x7f4f6c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f70a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c199820 0x7f4f6c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f70a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19a2d0 con 0x7f4f6c108680 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f70a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19a440 con 0x7f4f6c199820 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f70a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19a5b0 con 0x7f4f6c1035b0 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f69400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c108680 0x7f4f6c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58590/0 (socket says 172.21.15.6:58590) 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f69e00640 1 -- 172.21.15.6:0/3773111744 learned_addr learned my addr 172.21.15.6:0/3773111744 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.557+0000 7f4f6a800640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c199820 0x7f4f6c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69e00640 1 -- 172.21.15.6:0/3773111744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c199820 msgr2=0x7f4f6c199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69e00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c199820 0x7f4f6c199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69e00640 1 -- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c108680 msgr2=0x7f4f6c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69e00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c108680 0x7f4f6c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69e00640 1 -- 172.21.15.6:0/3773111744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f54009560 con 0x7f4f6c1035b0 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f6a800640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c199820 0x7f4f6c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69400640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c108680 0x7f4f6c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.558+0000 7f4f69e00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c19f6b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c036040 tx=0x7f4f5c033610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.559+0000 7f4f62a00640 1 -- 172.21.15.6:0/3773111744 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c003d80 con 0x7f4f6c1035b0 2024-02-26T06:28:27.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.559+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f64009490 con 0x7f4f6c1035b0 2024-02-26T06:28:27.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.559+0000 7f4f62a00640 1 -- 172.21.15.6:0/3773111744 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f5c02ce40 con 0x7f4f6c1035b0 2024-02-26T06:28:27.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.559+0000 7f4f62a00640 1 -- 172.21.15.6:0/3773111744 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0348f0 con 0x7f4f6c1035b0 2024-02-26T06:28:27.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.559+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c009840 con 0x7f4f6c1035b0 2024-02-26T06:28:27.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.562+0000 7f4f62a00640 1 -- 172.21.15.6:0/3773111744 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f4f5c0045b0 con 0x7f4f6c1035b0 2024-02-26T06:28:27.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.562+0000 7f4f739b8740 1 Processor -- start 2024-02-26T06:28:27.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 -- start start 2024-02-26T06:28:27.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de0b11e0 con 0x55f3de1433d0 2024-02-26T06:28:27.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de0ae710 con 0x55f3de154920 2024-02-26T06:28:27.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.563+0000 7f4f739b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de0a9ef0 con 0x55f3de142a20 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f6a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f6a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58596/0 (socket says 172.21.15.6:58596) 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48446/0 (socket says 172.21.15.6:48446) 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f6a800640 1 -- 172.21.15.6:0/2814668558 learned_addr learned my addr 172.21.15.6:0/2814668558 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69e00640 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f62a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4f500a22c0 0x7f4f500a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 -- 172.21.15.6:0/2814668558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 msgr2=0x55f3de142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 -- 172.21.15.6:0/2814668558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 msgr2=0x55f3de1543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de1543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69400640 1 -- 172.21.15.6:0/2814668558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3de06bdc0 con 0x55f3de154920 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.564+0000 7f4f69e00640 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.565+0000 7f4f62a00640 1 -- 172.21.15.6:0/3773111744 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4f5c0f0700 con 0x7f4f6c1035b0 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.565+0000 7f4f6a800640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4f500a22c0 0x7f4f500a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.565+0000 7f4f69400640 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de156d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009530 tx=0x7f4f54011a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33efcf3eba86d86 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.565+0000 7f4f5b000640 1 -- 172.21.15.6:0/2814668558 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400f640 con 0x55f3de154920 2024-02-26T06:28:27.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.565+0000 7f4f5b000640 1 -- 172.21.15.6:0/2814668558 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f5400fc80 con 0x55f3de154920 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.565+0000 7f4f5b000640 1 -- 172.21.15.6:0/2814668558 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54019dd0 con 0x55f3de154920 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 -- 172.21.15.6:0/2814668558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 msgr2=0x55f3de156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f6a800640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4f500a22c0 0x7f4f500a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f64010810 tx=0x7f4f640046c0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de156d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009530 tx=0x7f4f54011a60 comp rx=0 tx=0).stop 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 -- 172.21.15.6:0/2814668558 shutdown_connections 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de156d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2814668558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 -- 172.21.15.6:0/2814668558 >> 172.21.15.6:0/2814668558 conn(0x55f3de13d340 msgr2=0x55f3de13d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.566+0000 7f4f739b8740 1 -- 172.21.15.6:0/2814668558 shutdown_connections 2024-02-26T06:28:27.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.567+0000 7f4f739b8740 1 -- 172.21.15.6:0/2814668558 wait complete. 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.567+0000 7f4f739b8740 1 -- start start 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.567+0000 7f4f739b8740 1 Processor -- start 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de151990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de152410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de04f120 con 0x55f3de154920 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de04edc0 con 0x55f3de1433d0 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3dde181c0 con 0x55f3de142a20 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f739b8740 10 client.0 _ll_register_callbacks cb 0x55f3de09e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48450/0 (socket says 172.21.15.6:48450) 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f69400640 1 -- 172.21.15.6:0/2852204082 learned_addr learned my addr 172.21.15.6:0/2852204082 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f69e00640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.568+0000 7f4f6a800640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f69400640 1 -- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 msgr2=0x55f3de151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f69400640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f69400640 1 -- 172.21.15.6:0/2852204082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 msgr2=0x55f3de152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f69400640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f69400640 1 -- 172.21.15.6:0/2852204082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f54009df0 con 0x55f3de1433d0 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f69400640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de151ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4f54006010 tx=0x7f4f540197e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e0ec8dae148442 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f4f400640 1 -- 172.21.15.6:0/2852204082 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401adf0 con 0x55f3de1433d0 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3ddda97e0 con 0x55f3de1433d0 2024-02-26T06:28:27.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.569+0000 7f4f4f400640 1 -- 172.21.15.6:0/2852204082 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4f5400f640 con 0x55f3de1433d0 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f4f400640 1 -- 172.21.15.6:0/2852204082 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f5400f800 con 0x55f3de1433d0 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f739b8740 10 client.25625 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f3de0a8570 con 0x55f3de1433d0 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f4f400640 1 -- 172.21.15.6:0/2852204082 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f3ddda97e0 con 0x55f3de1433d0 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f4f400640 1 -- 172.21.15.6:0/2852204082 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4f540049c0 con 0x55f3de1433d0 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f739b8740 10 client.25625 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f739b8740 10 client.25625 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f739b8740 20 client.25625 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.570+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de164790 0x55f3de166bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.571+0000 7f4f739b8740 4 client.25625 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.571+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3ddde6720 con 0x55f3de164790 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.571+0000 7f4f69e00640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de164790 0x55f3de166bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.572+0000 7f4f69e00640 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de164790 0x55f3de166bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e54b7adf67b55ecb server_cookie=879f3fd431b445f0 in_seq=0 out_seq=0 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.572+0000 7f4f4f400640 10 client.25625 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.572+0000 7f4f4f400640 1 -- 172.21.15.6:0/2852204082 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3ddde6720 con 0x55f3de164790 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.572+0000 7f4f4f400640 10 client.25625 handle_command_reply: tid=0 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.572+0000 7f4f739b8740 1 client.25625 shutdown 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 msgr2=0x55f3de151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de151ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4f54006010 tx=0x7f4f540197e0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 shutdown_connections 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de164790 0x55f3de166bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de151ed0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 --2- 172.21.15.6:0/2852204082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f4f400640 0 client.25625 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:28:27.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 >> 172.21.15.6:0/2852204082 conn(0x55f3de13d340 msgr2=0x55f3de13f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 shutdown_connections 2024-02-26T06:28:27.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 1 -- 172.21.15.6:0/2852204082 wait complete. 2024-02-26T06:28:27.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.573+0000 7f4f739b8740 20 client.25625 trim_cache size 0 max 16384 2024-02-26T06:28:27.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.586+0000 7f4f739b8740 1 Processor -- start 2024-02-26T06:28:27.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.586+0000 7f4f739b8740 1 -- start start 2024-02-26T06:28:27.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.586+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.586+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de152230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.586+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f739b8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de16be20 con 0x55f3de1433d0 2024-02-26T06:28:27.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f739b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de142550 con 0x55f3de154920 2024-02-26T06:28:27.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f739b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de142350 con 0x55f3de142a20 2024-02-26T06:28:27.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58616/0 (socket says 172.21.15.6:58616) 2024-02-26T06:28:27.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 -- 172.21.15.6:0/3939318948 learned_addr learned my addr 172.21.15.6:0/3939318948 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69400640 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f6a800640 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 -- 172.21.15.6:0/3939318948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 msgr2=0x55f3de14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 -- 172.21.15.6:0/3939318948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 msgr2=0x55f3de152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.587+0000 7f4f69e00640 1 -- 172.21.15.6:0/3939318948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3de044a80 con 0x55f3de142a20 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.588+0000 7f4f69e00640 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de1540c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0b99e0 tx=0x7f4f5c009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b2e61d7577b5564 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.588+0000 7f4f5a600640 1 -- 172.21.15.6:0/3939318948 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0c1160 con 0x55f3de142a20 2024-02-26T06:28:27.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.588+0000 7f4f5a600640 1 -- 172.21.15.6:0/3939318948 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f5c0c57b0 con 0x55f3de142a20 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.588+0000 7f4f5a600640 1 -- 172.21.15.6:0/3939318948 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0edfc0 con 0x55f3de142a20 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 -- 172.21.15.6:0/3939318948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 msgr2=0x55f3de1540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de1540c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0b99e0 tx=0x7f4f5c009330 comp rx=0 tx=0).stop 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 -- 172.21.15.6:0/3939318948 shutdown_connections 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de154920 0x55f3de14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de1433d0 0x55f3de152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 --2- 172.21.15.6:0/3939318948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de1540c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 -- 172.21.15.6:0/3939318948 >> 172.21.15.6:0/3939318948 conn(0x55f3de13d340 msgr2=0x55f3de14cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 -- 172.21.15.6:0/3939318948 shutdown_connections 2024-02-26T06:28:27.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 -- 172.21.15.6:0/3939318948 wait complete. 2024-02-26T06:28:27.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.589+0000 7f4f739b8740 1 -- start start 2024-02-26T06:28:27.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 Processor -- start 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de1badf0 con 0x55f3de154920 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de1baf60 con 0x55f3de1433d0 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3de1bb120 con 0x55f3de142a20 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f739b8740 10 client.0 _ll_register_callbacks cb 0x7f4f54009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f6a800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f69400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48472/0 (socket says 172.21.15.6:48472) 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.590+0000 7f4f69400640 1 -- 172.21.15.6:0/1285182018 learned_addr learned my addr 172.21.15.6:0/1285182018 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f69400640 1 -- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 msgr2=0x55f3de15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f69400640 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f69400640 1 -- 172.21.15.6:0/1285182018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 msgr2=0x55f3de15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f69400640 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f69400640 1 -- 172.21.15.6:0/1285182018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c009840 con 0x55f3de1433d0 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f69400640 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de15a930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55f3de1511f0 tx=0x7f4f54005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f4112e115a4b2d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f4f400640 1 -- 172.21.15.6:0/1285182018 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400fa50 con 0x55f3de1433d0 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.591+0000 7f4f4f400640 1 -- 172.21.15.6:0/1285182018 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4f54004660 con 0x55f3de1433d0 2024-02-26T06:28:27.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.592+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f5400fc80 con 0x55f3de1433d0 2024-02-26T06:28:27.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.592+0000 7f4f4f400640 1 -- 172.21.15.6:0/1285182018 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f54004820 con 0x55f3de1433d0 2024-02-26T06:28:27.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.592+0000 7f4f739b8740 10 client.25631 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.592+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f3de06a620 con 0x55f3de1433d0 2024-02-26T06:28:27.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.592+0000 7f4f4f400640 1 -- 172.21.15.6:0/1285182018 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f5400fc80 con 0x55f3de1433d0 2024-02-26T06:28:27.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f4f400640 1 -- 172.21.15.6:0/1285182018 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4f5403e070 con 0x55f3de1433d0 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f739b8740 10 client.25631 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f739b8740 10 client.25631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f739b8740 20 client.25631 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f739b8740 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de1bc3d0 0x55f3de1be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f739b8740 4 client.25631 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3dde35890 con 0x55f3de1bc3d0 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.593+0000 7f4f69e00640 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de1bc3d0 0x55f3de1be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f69e00640 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de1bc3d0 0x55f3de1be7f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff45cd150222df0d server_cookie=ce6874589dbe4f4b in_seq=0 out_seq=0 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f4f400640 10 client.25631 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f4f400640 1 -- 172.21.15.6:0/1285182018 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3dde35890 con 0x55f3de1bc3d0 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f4f400640 10 client.25631 handle_command_reply: tid=0 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 client.25631 shutdown 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 msgr2=0x55f3de15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de15a930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55f3de1511f0 tx=0x7f4f54005e70 comp rx=0 tx=0).stop 2024-02-26T06:28:27.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 shutdown_connections 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x55f3de1bc3d0 0x55f3de1be7f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3de154920 0x55f3de15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3de1433d0 0x55f3de15a930 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 --2- 172.21.15.6:0/1285182018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3de142a20 0x55f3de15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 >> 172.21.15.6:0/1285182018 conn(0x55f3de13d340 msgr2=0x55f3de13e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f4f400640 0 client.25631 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 shutdown_connections 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.594+0000 7f4f739b8740 1 -- 172.21.15.6:0/1285182018 wait complete. 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f739b8740 20 client.25631 trim_cache size 0 max 16384 2024-02-26T06:28:27.596 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4f500a22c0 msgr2=0x7f4f500a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4f500a22c0 0x7f4f500a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f64010810 tx=0x7f4f640046c0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 msgr2=0x7f4f6c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c19f6b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c036040 tx=0x7f4f5c033610 comp rx=0 tx=0).stop 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 shutdown_connections 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4f500a22c0 0x7f4f500a4780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f6c199820 0x7f4f6c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f6c108680 0x7f4f6c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 --2- 172.21.15.6:0/3773111744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6c1035b0 0x7f4f6c19f6b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.595+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 >> 172.21.15.6:0/3773111744 conn(0x7f4f6c0fedf0 msgr2=0x7f4f6c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.596+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 shutdown_connections 2024-02-26T06:28:27.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.596+0000 7f4f70a00640 1 -- 172.21.15.6:0/3773111744 wait complete. 2024-02-26T06:28:27.611 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:27.611 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 -- 172.21.15.6:0/2895306798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073530 msgr2=0x7f1858073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 --2- 172.21.15.6:0/2895306798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073530 0x7f1858073910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802bcc0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 -- 172.21.15.6:0/2895306798 shutdown_connections 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 --2- 172.21.15.6:0/2895306798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f185810d820 0x7f185810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 --2- 172.21.15.6:0/2895306798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073e50 0x7f185810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 --2- 172.21.15.6:0/2895306798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073530 0x7f1858073910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 -- 172.21.15.6:0/2895306798 >> 172.21.15.6:0/2895306798 conn(0x7f18580fca90 msgr2=0x7f18580feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 -- 172.21.15.6:0/2895306798 shutdown_connections 2024-02-26T06:28:27.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.804+0000 7f1857200640 1 -- 172.21.15.6:0/2895306798 wait complete. 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.805+0000 7f1857200640 1 Processor -- start 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 -- start start 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 0x7f185819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073e50 0x7f185819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f185810d820 0x7f1858199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858199e20 con 0x7f1858073530 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858199f90 con 0x7f1858073e50 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1857200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185819a150 con 0x7f185810d820 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.806+0000 7f1855e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 0x7f185819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f184fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073e50 0x7f185819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 0x7f185819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54058/0 (socket says 172.21.15.6:54058) 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 -- 172.21.15.6:0/3103219842 learned_addr learned my addr 172.21.15.6:0/3103219842 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1856800640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f185810d820 0x7f1858199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 -- 172.21.15.6:0/3103219842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f185810d820 msgr2=0x7f1858199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f185810d820 0x7f1858199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 -- 172.21.15.6:0/3103219842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073e50 msgr2=0x7f185819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073e50 0x7f185819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 -- 172.21.15.6:0/3103219842 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840009490 con 0x7f1858073530 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.807+0000 7f1855e00640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 0x7f185819f580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1848000c00 tx=0x7f184802c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.808+0000 7f184ea00640 1 -- 172.21.15.6:0/3103219842 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184802dc50 con 0x7f1858073530 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.808+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1848009560 con 0x7f1858073530 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.808+0000 7f184ea00640 1 -- 172.21.15.6:0/3103219842 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f184802ddb0 con 0x7f1858073530 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.808+0000 7f184ea00640 1 -- 172.21.15.6:0/3103219842 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848003b80 con 0x7f1858073530 2024-02-26T06:28:27.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.808+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18581a6500 con 0x7f1858073530 2024-02-26T06:28:27.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.812+0000 7f184ea00640 1 -- 172.21.15.6:0/3103219842 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f18480054b0 con 0x7f1858073530 2024-02-26T06:28:27.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.813+0000 7f185ff7b740 1 Processor -- start 2024-02-26T06:28:27.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.813+0000 7f185ff7b740 1 -- start start 2024-02-26T06:28:27.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 0x55e055a251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a2e2c0 0x55e055a306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f185ff7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0559931e0 con 0x55e055a24dc0 2024-02-26T06:28:27.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f185ff7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055990710 con 0x55e055a256e0 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f185ff7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05598bef0 con 0x55e055a2e2c0 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f1856800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 0x55e055a251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f184ea00640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f183c0a22c0 0x7f183c0a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f1856800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 0x55e055a251a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54074/0 (socket says 172.21.15.6:54074) 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f1856800640 1 -- 172.21.15.6:0/2376567899 learned_addr learned my addr 172.21.15.6:0/2376567899 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f1855e00640 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.814+0000 7f184ea00640 1 -- 172.21.15.6:0/3103219842 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f18480f0840 con 0x7f1858073530 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f183c0a22c0 0x7f183c0a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f184fe00640 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a2e2c0 0x55e055a306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 -- 172.21.15.6:0/2376567899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a2e2c0 msgr2=0x55e055a306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a2e2c0 0x55e055a306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 -- 172.21.15.6:0/2376567899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 msgr2=0x55e055a2dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 -- 172.21.15.6:0/2376567899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e05594ddc0 con 0x55e055a24dc0 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1855e00640 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.815+0000 7f1856800640 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 0x55e055a251a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f18500139f0 tx=0x7f1850013eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dce5fe9b8bdde76 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.816+0000 7f1856800640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f183c0a22c0 0x7f183c0a4780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f185001b000 tx=0x7f185001d000 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.816+0000 7f1847400640 1 -- 172.21.15.6:0/2376567899 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185001c9e0 con 0x55e055a24dc0 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.816+0000 7f1847400640 1 -- 172.21.15.6:0/2376567899 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1850019070 con 0x55e055a24dc0 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.816+0000 7f1847400640 1 -- 172.21.15.6:0/2376567899 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185001de30 con 0x55e055a24dc0 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 -- 172.21.15.6:0/2376567899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 msgr2=0x55e055a251a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 0x55e055a251a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f18500139f0 tx=0x7f1850013eb0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 -- 172.21.15.6:0/2376567899 shutdown_connections 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a2e2c0 0x55e055a306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2376567899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a24dc0 0x55e055a251a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 -- 172.21.15.6:0/2376567899 >> 172.21.15.6:0/2376567899 conn(0x55e055a1f4f0 msgr2=0x55e055a1f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 -- 172.21.15.6:0/2376567899 shutdown_connections 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.817+0000 7f185ff7b740 1 -- 172.21.15.6:0/2376567899 wait complete. 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.818+0000 7f185ff7b740 1 -- start start 2024-02-26T06:28:27.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.818+0000 7f185ff7b740 1 Processor -- start 2024-02-26T06:28:27.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a2b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 0x55e055a2bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055931120 con 0x55e055a2e2c0 2024-02-26T06:28:27.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055930dc0 con 0x55e055a256e0 2024-02-26T06:28:27.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0556fa1c0 con 0x55e055a24dc0 2024-02-26T06:28:27.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f185ff7b740 10 client.0 _ll_register_callbacks cb 0x55e055980280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f1856800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 0x55e055a2bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f1856800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 0x55e055a2bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54076/0 (socket says 172.21.15.6:54076) 2024-02-26T06:28:27.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.819+0000 7f1856800640 1 -- 172.21.15.6:0/1520822919 learned_addr learned my addr 172.21.15.6:0/1520822919 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f1856800640 1 -- 172.21.15.6:0/1520822919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 msgr2=0x55e055a2b2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T06:28:27.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f1856800640 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a2b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f1856800640 1 -- 172.21.15.6:0/1520822919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 msgr2=0x55e055a2b7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:27.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f1856800640 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f1856800640 1 -- 172.21.15.6:0/1520822919 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1850013660 con 0x55e055a2e2c0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f1856800640 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 0x55e055a2bd30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1850011750 tx=0x7f185001cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafc1356eac6df52 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.820+0000 7f183be00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.821+0000 7f183be00640 1 -- 172.21.15.6:0/1520822919 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185001d3f0 con 0x55e055a2e2c0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.821+0000 7f183be00640 1 -- 172.21.15.6:0/1520822919 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f185001d550 con 0x55e055a2e2c0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.821+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e05568b7e0 con 0x55e055a2e2c0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.821+0000 7f183be00640 1 -- 172.21.15.6:0/1520822919 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f185001d710 con 0x55e055a2e2c0 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.821+0000 7f185ff7b740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.821+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e05598a570 con 0x55e055a2e2c0 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f183be00640 1 -- 172.21.15.6:0/1520822919 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1850003e50 con 0x55e055a2e2c0 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f183be00640 1 -- 172.21.15.6:0/1520822919 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1850024070 con 0x55e055a2e2c0 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f185ff7b740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f185ff7b740 10 client.15504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f185ff7b740 20 client.15504 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f185ff7b740 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a46a70 0x55e055a48ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f185ff7b740 4 client.15504 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e05568b7e0 con 0x55e055a46a70 2024-02-26T06:28:27.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.822+0000 7f1855e00640 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a46a70 0x55e055a48ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.823+0000 7f1855e00640 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a46a70 0x55e055a48ef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da98f854f74eb849 server_cookie=c7cf3312c90a984a in_seq=0 out_seq=0 2024-02-26T06:28:27.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.823+0000 7f183be00640 10 client.15504 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f183be00640 1 -- 172.21.15.6:0/1520822919 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e05568b7e0 con 0x55e055a46a70 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f183be00640 10 client.15504 handle_command_reply: tid=0 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 client.15504 shutdown 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 msgr2=0x55e055a2bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 0x55e055a2bd30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1850011750 tx=0x7f185001cea0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 shutdown_connections 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a46a70 0x55e055a48ef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a2e2c0 0x55e055a2bd30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a256e0 0x55e055a2b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 --2- 172.21.15.6:0/1520822919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a2b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f183be00640 0 client.15504 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:28:27.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.824+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 >> 172.21.15.6:0/1520822919 conn(0x55e055a1f4f0 msgr2=0x55e055a21240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.825+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 shutdown_connections 2024-02-26T06:28:27.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.825+0000 7f185ff7b740 1 -- 172.21.15.6:0/1520822919 wait complete. 2024-02-26T06:28:27.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.825+0000 7f185ff7b740 20 client.15504 trim_cache size 0 max 16384 2024-02-26T06:28:27.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 Processor -- start 2024-02-26T06:28:27.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 -- start start 2024-02-26T06:28:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a4d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a28510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a28a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055a2dd80 con 0x55e055a256e0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055a467f0 con 0x55e055a2e2c0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.849+0000 7f185ff7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055a26eb0 con 0x55e055a24dc0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a4d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1856800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a28a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a4d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58670/0 (socket says 172.21.15.6:58670) 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 -- 172.21.15.6:0/2435485707 learned_addr learned my addr 172.21.15.6:0/2435485707 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f184fe00640 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a28510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 -- 172.21.15.6:0/2435485707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 msgr2=0x55e055a28a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a28a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 -- 172.21.15.6:0/2435485707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 msgr2=0x55e055a28510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a28510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 -- 172.21.15.6:0/2435485707 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e055926a80 con 0x55e055a24dc0 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1856800640 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a28a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:27.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.850+0000 7f1855e00640 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a4d890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f18480bd0e0 tx=0x7f1848005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f7d57081dced3d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.851+0000 7f1846a00640 1 -- 172.21.15.6:0/2435485707 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480bd150 con 0x55e055a24dc0 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.851+0000 7f1846a00640 1 -- 172.21.15.6:0/2435485707 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18480c12a0 con 0x55e055a24dc0 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.851+0000 7f1846a00640 1 -- 172.21.15.6:0/2435485707 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848003e70 con 0x55e055a24dc0 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 -- 172.21.15.6:0/2435485707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 msgr2=0x55e055a4d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a4d890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f18480bd0e0 tx=0x7f1848005e20 comp rx=0 tx=0).stop 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 -- 172.21.15.6:0/2435485707 shutdown_connections 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a28a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a28510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 --2- 172.21.15.6:0/2435485707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a4d890 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 -- 172.21.15.6:0/2435485707 >> 172.21.15.6:0/2435485707 conn(0x55e055a1f4f0 msgr2=0x55e055a1a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 -- 172.21.15.6:0/2435485707 shutdown_connections 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 -- 172.21.15.6:0/2435485707 wait complete. 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.852+0000 7f185ff7b740 1 -- start start 2024-02-26T06:28:27.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 Processor -- start 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a3cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a3d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a96130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055a96670 con 0x55e055a256e0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055a967e0 con 0x55e055a2e2c0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e055a969a0 con 0x55e055a24dc0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f185ff7b740 10 client.0 _ll_register_callbacks cb 0x7f18500097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f1855e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a3cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f184fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a3d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f1855e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a3cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58684/0 (socket says 172.21.15.6:58684) 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.853+0000 7f1855e00640 1 -- 172.21.15.6:0/3925434007 learned_addr learned my addr 172.21.15.6:0/3925434007 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f1856800640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a96130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f184fe00640 1 -- 172.21.15.6:0/3925434007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 msgr2=0x55e055a3cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f184fe00640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a3cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f184fe00640 1 -- 172.21.15.6:0/3925434007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 msgr2=0x55e055a96130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f184fe00640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a96130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f184fe00640 1 -- 172.21.15.6:0/3925434007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1848009560 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f1855e00640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a3cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.854+0000 7f184fe00640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a3d4d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e055a1ad40 tx=0x7f1840010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96778a12aa6a09e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f183be00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f183be00640 1 -- 172.21.15.6:0/3925434007 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840004250 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f183be00640 1 -- 172.21.15.6:0/3925434007 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f18400043b0 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1850019070 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f183be00640 1 -- 172.21.15.6:0/3925434007 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1840005020 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f185ff7b740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e05594c620 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.855+0000 7f183be00640 1 -- 172.21.15.6:0/3925434007 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1840029020 con 0x55e055a256e0 2024-02-26T06:28:27.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f183be00640 1 -- 172.21.15.6:0/3925434007 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f18400047a0 con 0x55e055a256e0 2024-02-26T06:28:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f185ff7b740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f185ff7b740 10 client.15510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-02-26T06:28:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f185ff7b740 20 client.15510 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f185ff7b740 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a9f4e0 0x55e055aa1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f185ff7b740 4 client.15510 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e0556c8720 con 0x55e055a9f4e0 2024-02-26T06:28:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.856+0000 7f1855e00640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a9f4e0 0x55e055aa1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f1855e00640 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a9f4e0 0x55e055aa1900 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fa4c059f40e0a20 server_cookie=41bcb74812581cd7 in_seq=0 out_seq=0 2024-02-26T06:28:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f183be00640 10 client.15510 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:28:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f183be00640 1 -- 172.21.15.6:0/3925434007 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e0556c8720 con 0x55e055a9f4e0 2024-02-26T06:28:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f183be00640 10 client.15510 handle_command_reply: tid=0 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 client.15510 shutdown 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 msgr2=0x55e055a3d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a3d4d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e055a1ad40 tx=0x7f1840010b40 comp rx=0 tx=0).stop 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 shutdown_connections 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55e055a9f4e0 0x55e055aa1900 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e055a2e2c0 0x55e055a96130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e055a256e0 0x55e055a3d4d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 --2- 172.21.15.6:0/3925434007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e055a24dc0 0x55e055a3cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.857+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 >> 172.21.15.6:0/3925434007 conn(0x55e055a1f4f0 msgr2=0x55e055a29740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.858+0000 7f183be00640 0 client.15510 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.858+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 shutdown_connections 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.858+0000 7f185ff7b740 1 -- 172.21.15.6:0/3925434007 wait complete. 2024-02-26T06:28:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.858+0000 7f185ff7b740 20 client.15510 trim_cache size 0 max 16384 2024-02-26T06:28:27.860 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:28:27.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f183c0a22c0 msgr2=0x7f183c0a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f183c0a22c0 0x7f183c0a4780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f185001b000 tx=0x7f185001d000 comp rx=0 tx=0).stop 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 msgr2=0x7f185819f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 0x7f185819f580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1848000c00 tx=0x7f184802c730 comp rx=0 tx=0).stop 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 shutdown_connections 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f183c0a22c0 0x7f183c0a4780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f185810d820 0x7f1858199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1858073e50 0x7f185819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 --2- 172.21.15.6:0/3103219842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1858073530 0x7f185819f580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.859+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 >> 172.21.15.6:0/3103219842 conn(0x7f18580fca90 msgr2=0x7f18580fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.860+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 shutdown_connections 2024-02-26T06:28:27.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:27.860+0000 7f1857200640 1 -- 172.21.15.6:0/3103219842 wait complete. 2024-02-26T06:28:27.877 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:28:27.877 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:28:27.936 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:27 smithi006 ceph-mon[31790]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:28:28.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 -- 172.21.15.6:0/462065047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1062a0 msgr2=0x7f5ebc10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 --2- 172.21.15.6:0/462065047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1062a0 0x7f5ebc10de70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009c10 tx=0x7f5eb402bee0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 -- 172.21.15.6:0/462065047 shutdown_connections 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 --2- 172.21.15.6:0/462065047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1062a0 0x7f5ebc10de70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 --2- 172.21.15.6:0/462065047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ebc101b80 0x7f5ebc105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 --2- 172.21.15.6:0/462065047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 -- 172.21.15.6:0/462065047 >> 172.21.15.6:0/462065047 conn(0x7f5ebc0fca90 msgr2=0x7f5ebc0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 -- 172.21.15.6:0/462065047 shutdown_connections 2024-02-26T06:28:28.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.054+0000 7f5ec2600640 1 -- 172.21.15.6:0/462065047 wait complete. 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.055+0000 7f5ec2600640 1 Processor -- start 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.055+0000 7f5ec2600640 1 -- start start 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec2600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ebc1062a0 0x7f5ebc19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec2600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ebc199600 0x7f5ebc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec2600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc199ff0 con 0x7f5ebc1011d0 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc19a160 con 0x7f5ebc1062a0 2024-02-26T06:28:28.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec2600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc19a320 con 0x7f5ebc199600 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54110/0 (socket says 172.21.15.6:54110) 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec1200640 1 -- 172.21.15.6:0/2485594103 learned_addr learned my addr 172.21.15.6:0/2485594103 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ebc1062a0 0x7f5ebc19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.056+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ebc199600 0x7f5ebc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec1200640 1 -- 172.21.15.6:0/2485594103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ebc199600 msgr2=0x7f5ebc199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec1200640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ebc199600 0x7f5ebc199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec1200640 1 -- 172.21.15.6:0/2485594103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ebc1062a0 msgr2=0x7f5ebc19f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec1200640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ebc1062a0 0x7f5ebc19f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec1200640 1 -- 172.21.15.6:0/2485594103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009560 con 0x7f5ebc1011d0 2024-02-26T06:28:28.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec1200640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc19f450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000e9c0 tx=0x7f5eb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:28.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5eb9600640 1 -- 172.21.15.6:0/2485594103 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000cde0 con 0x7f5ebc1011d0 2024-02-26T06:28:28.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5eb9600640 1 -- 172.21.15.6:0/2485594103 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb00035f0 con 0x7f5ebc1011d0 2024-02-26T06:28:28.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.057+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb4009840 con 0x7f5ebc1011d0 2024-02-26T06:28:28.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.058+0000 7f5eb9600640 1 -- 172.21.15.6:0/2485594103 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0010670 con 0x7f5ebc1011d0 2024-02-26T06:28:28.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.058+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ebc1a66e0 con 0x7f5ebc1011d0 2024-02-26T06:28:28.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 Processor -- start 2024-02-26T06:28:28.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 -- start start 2024-02-26T06:28:28.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3c951e0 con 0x5636e3d302c0 2024-02-26T06:28:28.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.061+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3c92710 con 0x5636e3d276e0 2024-02-26T06:28:28.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3c8def0 con 0x5636e3d26dc0 2024-02-26T06:28:28.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ebbe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ebbe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d326b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54118/0 (socket says 172.21.15.6:54118) 2024-02-26T06:28:28.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ebbe00640 1 -- 172.21.15.6:0/3425540747 learned_addr learned my addr 172.21.15.6:0/3425540747 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ec1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ebbe00640 1 -- 172.21.15.6:0/3425540747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 msgr2=0x5636e3d271a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.062+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d271a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5ebbe00640 1 -- 172.21.15.6:0/3425540747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 msgr2=0x5636e3d2fd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:28:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5ebbe00640 1 -- 172.21.15.6:0/3425540747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e3c4fdc0 con 0x5636e3d302c0 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d326b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009ba0 tx=0x7f5ea8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1fe0590f067104 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5eae000640 1 -- 172.21.15.6:0/3425540747 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004230 con 0x5636e3d302c0 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5eb9600640 1 -- 172.21.15.6:0/2485594103 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f5eb0003ad0 con 0x7f5ebc1011d0 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5eae000640 1 -- 172.21.15.6:0/3425540747 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea8009e10 con 0x5636e3d302c0 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.063+0000 7f5eae000640 1 -- 172.21.15.6:0/3425540747 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004ca0 con 0x5636e3d302c0 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3425540747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 msgr2=0x5636e3d326b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d326b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009ba0 tx=0x7f5ea8011810 comp rx=0 tx=0).stop 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3425540747 shutdown_connections 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d326b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3425540747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3425540747 >> 172.21.15.6:0/3425540747 conn(0x5636e3d214f0 msgr2=0x5636e3d218d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3425540747 shutdown_connections 2024-02-26T06:28:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.064+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3425540747 wait complete. 2024-02-26T06:28:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.065+0000 7f5ec6a6b740 1 -- start start 2024-02-26T06:28:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.065+0000 7f5ec6a6b740 1 Processor -- start 2024-02-26T06:28:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5eb9600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea40a22d0 0x7f5ea40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d2d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5eb9600640 1 -- 172.21.15.6:0/2485594103 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5eb0014070 con 0x7f5ebc1011d0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3c33120 con 0x5636e3d302c0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3c32dc0 con 0x5636e3d276e0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e39fc1c0 con 0x5636e3d26dc0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec6a6b740 10 client.0 _ll_register_callbacks cb 0x5636e3c82280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d2d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d2d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58714/0 (socket says 172.21.15.6:58714) 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.066+0000 7f5ec1200640 1 -- 172.21.15.6:0/2782103441 learned_addr learned my addr 172.21.15.6:0/2782103441 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1200640 1 -- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 msgr2=0x5636e3d2d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1200640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1200640 1 -- 172.21.15.6:0/2782103441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 msgr2=0x5636e3d2dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1200640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d2dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1200640 1 -- 172.21.15.6:0/2782103441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea8009850 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d2dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea40a22d0 0x7f5ea40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ec1200640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d2d320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5eb009ab40 tx=0x7f5eb009aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6afd7c8aa7b1c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ea2a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.067+0000 7f5ea2a00640 1 -- 172.21.15.6:0/2782103441 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb009ae40 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636e398d7e0 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ea2a00640 1 -- 172.21.15.6:0/2782103441 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5eb009ef90 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ea2a00640 1 -- 172.21.15.6:0/2782103441 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb00a35e0 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ec6a6b740 10 client.25636 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5636e3c8c570 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ea2a00640 1 -- 172.21.15.6:0/2782103441 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5636e398d7e0 con 0x5636e3d26dc0 2024-02-26T06:28:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ea2a00640 1 -- 172.21.15.6:0/2782103441 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5eb001f060 con 0x5636e3d26dc0 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.068+0000 7f5ec6a6b740 10 client.25636 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ec6a6b740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ec6a6b740 20 client.25636 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3d48a90 0x5636e3d4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ec6a6b740 4 client.25636 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636e39ca720 con 0x5636e3d48a90 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea40a22d0 0x7f5ea40a4790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4036040 tx=0x7f5eb4009590 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.069+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3d48a90 0x5636e3d4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.070+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3d48a90 0x5636e3d4ae80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311cbaf8e5fc2ca7 server_cookie=b30f3ff79026e8e7 in_seq=0 out_seq=0 2024-02-26T06:28:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.070+0000 7f5ea2a00640 10 client.25636 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ea2a00640 1 -- 172.21.15.6:0/2782103441 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636e39ca720 con 0x5636e3d48a90 2024-02-26T06:28:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ea2a00640 10 client.25636 handle_command_reply: tid=0 2024-02-26T06:28:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 client.25636 shutdown 2024-02-26T06:28:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 msgr2=0x5636e3d2d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d2d320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5eb009ab40 tx=0x7f5eb009aa70 comp rx=0 tx=0).stop 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 shutdown_connections 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3d48a90 0x5636e3d4ae80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d302c0 0x5636e3d2dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d276e0 0x5636e3d2d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ea2a00640 0 client.25636 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/2782103441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d26dc0 0x5636e3d2d320 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 >> 172.21.15.6:0/2782103441 conn(0x5636e3d214f0 msgr2=0x5636e3d23240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 shutdown_connections 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/2782103441 wait complete. 2024-02-26T06:28:28.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.071+0000 7f5ec6a6b740 20 client.25636 trim_cache size 0 max 16384 2024-02-26T06:28:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 Processor -- start 2024-02-26T06:28:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 -- start start 2024-02-26T06:28:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 0x5636e3d2fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d276e0 0x5636e3d285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d302c0 0x5636e3d28ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3d3ce70 con 0x5636e3d276e0 2024-02-26T06:28:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3d224f0 con 0x5636e3d26dc0 2024-02-26T06:28:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.091+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3d22720 con 0x5636e3d302c0 2024-02-26T06:28:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 0x5636e3d2fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 0x5636e3d2fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48550/0 (socket says 172.21.15.6:48550) 2024-02-26T06:28:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 -- 172.21.15.6:0/822665948 learned_addr learned my addr 172.21.15.6:0/822665948 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d302c0 0x5636e3d28ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d276e0 0x5636e3d285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 -- 172.21.15.6:0/822665948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d302c0 msgr2=0x5636e3d28ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d302c0 0x5636e3d28ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 -- 172.21.15.6:0/822665948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d276e0 msgr2=0x5636e3d285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d276e0 0x5636e3d285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ec1200640 1 -- 172.21.15.6:0/822665948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e3c28a80 con 0x5636e3d26dc0 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.092+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d276e0 0x5636e3d285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.093+0000 7f5ec1200640 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 0x5636e3d2fa50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0098560 tx=0x7f5eb0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f936122ab4f11332 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.093+0000 7f5ead600640 1 -- 172.21.15.6:0/822665948 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003760 con 0x5636e3d26dc0 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.093+0000 7f5ead600640 1 -- 172.21.15.6:0/822665948 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb00aee90 con 0x5636e3d26dc0 2024-02-26T06:28:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.093+0000 7f5ead600640 1 -- 172.21.15.6:0/822665948 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00b93d0 con 0x5636e3d26dc0 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/822665948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 msgr2=0x5636e3d2fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 0x5636e3d2fa50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0098560 tx=0x7f5eb0000bc0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/822665948 shutdown_connections 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d302c0 0x5636e3d28ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d276e0 0x5636e3d285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/822665948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d26dc0 0x5636e3d2fa50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/822665948 >> 172.21.15.6:0/822665948 conn(0x5636e3d214f0 msgr2=0x5636e3d1c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/822665948 shutdown_connections 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/822665948 wait complete. 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 -- start start 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.094+0000 7f5ec6a6b740 1 Processor -- start 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 0x5636e3d35e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d276e0 0x5636e3d363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d302c0 0x5636e3d36910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3d9f100 con 0x5636e3d26dc0 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3d9f270 con 0x5636e3d302c0 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e3d9f430 con 0x5636e3d276e0 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec6a6b740 10 client.0 _ll_register_callbacks cb 0x7f5ea80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 0x5636e3d35e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 0x5636e3d35e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54144/0 (socket says 172.21.15.6:54144) 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ec1200640 1 -- 172.21.15.6:0/3024054682 learned_addr learned my addr 172.21.15.6:0/3024054682 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.095+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d276e0 0x5636e3d363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1c00640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d302c0 0x5636e3d36910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1200640 1 -- 172.21.15.6:0/3024054682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d276e0 msgr2=0x5636e3d363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1200640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d276e0 0x5636e3d363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1200640 1 -- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d302c0 msgr2=0x5636e3d36910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1200640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d302c0 0x5636e3d36910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1200640 1 -- 172.21.15.6:0/3024054682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e3c8c570 con 0x5636e3d26dc0 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec1200640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 0x5636e3d35e90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00985e0 tx=0x7f5eb00b92b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90dd522f2133fb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ea2a00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ea2a00640 1 -- 172.21.15.6:0/3024054682 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00b93b0 con 0x5636e3d26dc0 2024-02-26T06:28:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ea2a00640 1 -- 172.21.15.6:0/3024054682 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5eb00a35e0 con 0x5636e3d26dc0 2024-02-26T06:28:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.096+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea8009e10 con 0x5636e3d26dc0 2024-02-26T06:28:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ea2a00640 1 -- 172.21.15.6:0/3024054682 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb0003760 con 0x5636e3d26dc0 2024-02-26T06:28:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5636e3c4e620 con 0x5636e3d26dc0 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ea2a00640 1 -- 172.21.15.6:0/3024054682 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea8009e10 con 0x5636e3d26dc0 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ea2a00640 1 -- 172.21.15.6:0/3024054682 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5eb001f030 con 0x5636e3d26dc0 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 20 client.15525 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3da0720 0x5636e3da2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 4 client.15525 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.097+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636e3a19890 con 0x5636e3da0720 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.098+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3da0720 0x5636e3da2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.098+0000 7f5ebbe00640 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3da0720 0x5636e3da2b10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a47e180612c4d41 server_cookie=5aff17c3d6e78e98 in_seq=0 out_seq=0 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.098+0000 7f5ea2a00640 10 client.15525 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ea2a00640 1 -- 172.21.15.6:0/3024054682 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636e3a19890 con 0x5636e3da0720 2024-02-26T06:28:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ea2a00640 10 client.15525 handle_command_reply: tid=0 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 client.15525 shutdown 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 msgr2=0x5636e3d35e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 0x5636e3d35e90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00985e0 tx=0x7f5eb00b92b0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 shutdown_connections 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5636e3da0720 0x5636e3da2b10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636e3d302c0 0x5636e3d36910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e3d276e0 0x5636e3d363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ea2a00640 0 client.15525 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 --2- 172.21.15.6:0/3024054682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5636e3d26dc0 0x5636e3d35e90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 >> 172.21.15.6:0/3024054682 conn(0x5636e3d214f0 msgr2=0x5636e3d285a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 shutdown_connections 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 1 -- 172.21.15.6:0/3024054682 wait complete. 2024-02-26T06:28:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.099+0000 7f5ec6a6b740 20 client.15525 trim_cache size 0 max 16384 2024-02-26T06:28:28.102 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea40a22d0 msgr2=0x7f5ea40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea40a22d0 0x7f5ea40a4790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4036040 tx=0x7f5eb4009590 comp rx=0 tx=0).stop 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 msgr2=0x7f5ebc19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc19f450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000e9c0 tx=0x7f5eb000ee80 comp rx=0 tx=0).stop 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 shutdown_connections 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5ea40a22d0 0x7f5ea40a4790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ebc199600 0x7f5ebc199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ebc1062a0 0x7f5ebc19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 --2- 172.21.15.6:0/2485594103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc1011d0 0x7f5ebc19f450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.100+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 >> 172.21.15.6:0/2485594103 conn(0x7f5ebc0fca90 msgr2=0x7f5ebc10c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.101+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 shutdown_connections 2024-02-26T06:28:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:28.101+0000 7f5ec2600640 1 -- 172.21.15.6:0/2485594103 wait complete. 2024-02-26T06:28:28.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:27 smithi090 ceph-mon[35681]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:28:28.160 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:27 smithi167 ceph-mon[36229]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:28:29.117 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:28:29.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:28:29.117 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:28:29.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.338+0000 7f030c200640 1 -- 172.21.15.6:0/427405864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304103f40 msgr2=0x7f0304107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.338+0000 7f030c200640 1 --2- 172.21.15.6:0/427405864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304103f40 0x7f0304107f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f002bc30 comp rx=0 tx=0).stop 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 -- 172.21.15.6:0/427405864 shutdown_connections 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 --2- 172.21.15.6:0/427405864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0304108660 0x7f0304110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 --2- 172.21.15.6:0/427405864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304103f40 0x7f0304107f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 --2- 172.21.15.6:0/427405864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0304103590 0x7f0304103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 -- 172.21.15.6:0/427405864 >> 172.21.15.6:0/427405864 conn(0x7f03040fedd0 msgr2=0x7f03041011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 -- 172.21.15.6:0/427405864 shutdown_connections 2024-02-26T06:28:29.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.339+0000 7f030c200640 1 -- 172.21.15.6:0/427405864 wait complete. 2024-02-26T06:28:29.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.340+0000 7f030c200640 1 Processor -- start 2024-02-26T06:28:29.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.340+0000 7f030c200640 1 -- start start 2024-02-26T06:28:29.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f030c200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0304103590 0x7f030419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f030c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 0x7f030419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f030c200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0304199830 0x7f0304199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f030c200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030419a2e0 con 0x7f0304103590 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f030c200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030419a450 con 0x7f0304199830 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f030c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030419a5c0 con 0x7f0304108660 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f0308e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 0x7f030419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f0309800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0304103590 0x7f030419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f0308e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 0x7f030419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58774/0 (socket says 172.21.15.6:58774) 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.341+0000 7f0308e00640 1 -- 172.21.15.6:0/1616834792 learned_addr learned my addr 172.21.15.6:0/1616834792 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f030a200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0304199830 0x7f0304199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f0308e00640 1 -- 172.21.15.6:0/1616834792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0304199830 msgr2=0x7f0304199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f0308e00640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0304199830 0x7f0304199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f0308e00640 1 -- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0304103590 msgr2=0x7f030419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f0308e00640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0304103590 0x7f030419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f0308e00640 1 -- 172.21.15.6:0/1616834792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009560 con 0x7f0304108660 2024-02-26T06:28:29.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.342+0000 7f0308e00640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 0x7f030419fbb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000c00 tx=0x7f02f0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.343+0000 7f0302000640 1 -- 172.21.15.6:0/1616834792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f002dea0 con 0x7f0304108660 2024-02-26T06:28:29.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.343+0000 7f0302000640 1 -- 172.21.15.6:0/1616834792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02f0004440 con 0x7f0304108660 2024-02-26T06:28:29.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.343+0000 7f0302000640 1 -- 172.21.15.6:0/1616834792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0003cc0 con 0x7f0304108660 2024-02-26T06:28:29.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.343+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03041a64c0 con 0x7f0304108660 2024-02-26T06:28:29.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.343+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03041a6890 con 0x7f0304108660 2024-02-26T06:28:29.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.349+0000 7f030f1e3740 1 Processor -- start 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.349+0000 7f030f1e3740 1 -- start start 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.349+0000 7f0302000640 1 -- 172.21.15.6:0/1616834792 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f02f0004df0 con 0x7f0304108660 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.349+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030f1e3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a9df2d0 con 0x55665aa2d410 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030f1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a991e20 con 0x55665aa24830 2024-02-26T06:28:29.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030f1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a98f350 con 0x55665aa23f10 2024-02-26T06:28:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f0308e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58784/0 (socket says 172.21.15.6:58784) 2024-02-26T06:28:29.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f030a200640 1 -- 172.21.15.6:0/1743738692 learned_addr learned my addr 172.21.15.6:0/1743738692 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.350+0000 7f0309800640 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0309800640 1 -- 172.21.15.6:0/1743738692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 msgr2=0x55665aa242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0309800640 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0309800640 1 -- 172.21.15.6:0/1743738692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 msgr2=0x55665aa2f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0309800640 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0309800640 1 -- 172.21.15.6:0/1743738692 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665a953c50 con 0x55665aa24830 2024-02-26T06:28:29.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0309800640 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ced0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02f800bbe0 tx=0x7f02f80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7ef484bfd32ab0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f02f6a00640 1 -- 172.21.15.6:0/1743738692 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004030 con 0x55665aa24830 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.351+0000 7f0302000640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f02ec0a2340 0x7f02ec0a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.352+0000 7f0302000640 1 -- 172.21.15.6:0/1616834792 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f02f00f3170 con 0x7f0304108660 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.352+0000 7f02f6a00640 1 -- 172.21.15.6:0/1743738692 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02f8015070 con 0x55665aa24830 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.352+0000 7f02f6a00640 1 -- 172.21.15.6:0/1743738692 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80116a0 con 0x55665aa24830 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.352+0000 7f030a200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f02ec0a2340 0x7f02ec0a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 -- 172.21.15.6:0/1743738692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 msgr2=0x55665aa2ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ced0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02f800bbe0 tx=0x7f02f80107a0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030a200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f02ec0a2340 0x7f02ec0a4800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005f40 tx=0x7f02fc009600 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 -- 172.21.15.6:0/1743738692 shutdown_connections 2024-02-26T06:28:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ced0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1743738692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 -- 172.21.15.6:0/1743738692 >> 172.21.15.6:0/1743738692 conn(0x55665aa1e640 msgr2=0x55665aa1ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 -- 172.21.15.6:0/1743738692 shutdown_connections 2024-02-26T06:28:29.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.353+0000 7f030f1e3740 1 -- 172.21.15.6:0/1743738692 wait complete. 2024-02-26T06:28:29.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.354+0000 7f030f1e3740 1 -- start start 2024-02-26T06:28:29.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.354+0000 7f030f1e3740 1 Processor -- start 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.354+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa2a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa2a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030f1e3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a98ab30 con 0x55665aa2d410 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030f1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a92ffa0 con 0x55665aa23f10 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030f1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a92fc40 con 0x55665aa24830 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030f1e3740 10 client.0 _ll_register_callbacks cb 0x55665a69bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa2a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa2a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48608/0 (socket says 172.21.15.6:48608) 2024-02-26T06:28:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 -- 172.21.15.6:0/1988118553 learned_addr learned my addr 172.21.15.6:0/1988118553 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0308e00640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa2a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030a200640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 -- 172.21.15.6:0/1988118553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 msgr2=0x55665aa2a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa2a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 -- 172.21.15.6:0/1988118553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 msgr2=0x55665aa2ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0309800640 1 -- 172.21.15.6:0/1988118553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f800b8a0 con 0x55665aa23f10 2024-02-26T06:28:29.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f0308e00640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa2a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:29.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.355+0000 7f030a200640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:29.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.356+0000 7f0309800640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa2a3b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02f800bbb0 tx=0x7f02f800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd7c3582958458a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.356+0000 7f02eb400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.356+0000 7f02eb400640 1 -- 172.21.15.6:0/1988118553 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f801f070 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.356+0000 7f02eb400640 1 -- 172.21.15.6:0/1988118553 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f02f8017070 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.356+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55665a68a860 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f02eb400640 1 -- 172.21.15.6:0/1988118553 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f8005250 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 10 client.25673 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55665a9891b0 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f02eb400640 1 -- 172.21.15.6:0/1988118553 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02f8032020 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f02eb400640 1 -- 172.21.15.6:0/1988118553 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02f8004920 con 0x55665aa23f10 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 10 client.25673 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:29.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 20 client.25673 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa3fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 4 client.25673 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.357+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55665a6db410 con 0x55665aa3d700 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.358+0000 7f0308e00640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa3fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.358+0000 7f0308e00640 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa3fb20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9350bb58e55fdb8 server_cookie=980143fc962a440 in_seq=0 out_seq=0 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.358+0000 7f02eb400640 10 client.25673 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.359+0000 7f02eb400640 1 -- 172.21.15.6:0/1988118553 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55665a6db410 con 0x55665aa3d700 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.359+0000 7f02eb400640 10 client.25673 handle_command_reply: tid=0 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.359+0000 7f030f1e3740 1 client.25673 shutdown 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.359+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 msgr2=0x55665aa2a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.359+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa2a3b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02f800bbb0 tx=0x7f02f800bfc0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 shutdown_connections 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa3fb20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f02eb400640 0 client.25673 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa2a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1988118553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa2a3b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 >> 172.21.15.6:0/1988118553 conn(0x55665aa1e640 msgr2=0x55665aa20390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 shutdown_connections 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 1 -- 172.21.15.6:0/1988118553 wait complete. 2024-02-26T06:28:29.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.360+0000 7f030f1e3740 20 client.25673 trim_cache size 0 max 16384 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.376+0000 7f030f1e3740 1 Processor -- start 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 -- start start 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa2cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa27630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665a6d30c0 con 0x55665aa2d410 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665aa23b70 con 0x55665aa24830 2024-02-26T06:28:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030f1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665aa1f640 con 0x55665aa23f10 2024-02-26T06:28:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f0308e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f0308e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48612/0 (socket says 172.21.15.6:48612) 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f030a200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa27630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f0308e00640 1 -- 172.21.15.6:0/1100050626 learned_addr learned my addr 172.21.15.6:0/1100050626 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.377+0000 7f0309800640 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa2cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f0308e00640 1 -- 172.21.15.6:0/1100050626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 msgr2=0x55665aa2cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f0308e00640 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa2cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f0308e00640 1 -- 172.21.15.6:0/1100050626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 msgr2=0x55665aa27630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f0308e00640 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa27630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f0308e00640 1 -- 172.21.15.6:0/1100050626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665a925a60 con 0x55665aa24830 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f0308e00640 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ac50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f02f0005ec0 tx=0x7f02f00dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d91188d2b93412 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f02f6000640 1 -- 172.21.15.6:0/1100050626 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00bc1d0 con 0x55665aa24830 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f02f6000640 1 -- 172.21.15.6:0/1100050626 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02f00bbf40 con 0x55665aa24830 2024-02-26T06:28:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.378+0000 7f02f6000640 1 -- 172.21.15.6:0/1100050626 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00bc400 con 0x55665aa24830 2024-02-26T06:28:29.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 -- 172.21.15.6:0/1100050626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 msgr2=0x55665aa2ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ac50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f02f0005ec0 tx=0x7f02f00dc040 comp rx=0 tx=0).stop 2024-02-26T06:28:29.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 -- 172.21.15.6:0/1100050626 shutdown_connections 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa27630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa24830 0x55665aa2ac50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 --2- 172.21.15.6:0/1100050626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa23f10 0x55665aa2cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.379+0000 7f030f1e3740 1 -- 172.21.15.6:0/1100050626 >> 172.21.15.6:0/1100050626 conn(0x55665aa1e640 msgr2=0x55665aa28c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.380+0000 7f030f1e3740 1 -- 172.21.15.6:0/1100050626 shutdown_connections 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.380+0000 7f030f1e3740 1 -- 172.21.15.6:0/1100050626 wait complete. 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.380+0000 7f030f1e3740 1 -- start start 2024-02-26T06:28:29.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.380+0000 7f030f1e3740 1 Processor -- start 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665aa93c50 con 0x55665aa2d410 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665aa93d90 con 0x55665aa23f10 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665aa93ed0 con 0x55665aa24830 2024-02-26T06:28:29.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030f1e3740 10 client.0 _ll_register_callbacks cb 0x7f02f800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f0309800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f0309800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa32d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48622/0 (socket says 172.21.15.6:48622) 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f0309800640 1 -- 172.21.15.6:0/2482375269 learned_addr learned my addr 172.21.15.6:0/2482375269 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f030a200640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.381+0000 7f0308e00640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f0309800640 1 -- 172.21.15.6:0/2482375269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 msgr2=0x55665aa33290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f0309800640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa33290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f0309800640 1 -- 172.21.15.6:0/2482375269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 msgr2=0x55665aa337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f0309800640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f0309800640 1 -- 172.21.15.6:0/2482375269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03041a6890 con 0x55665aa23f10 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f0309800640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa32d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f02f8003b70 tx=0x7f02f8004fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edb18e989176022 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f02eb400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:29.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f02eb400640 1 -- 172.21.15.6:0/2482375269 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f801f040 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f02eb400640 1 -- 172.21.15.6:0/2482375269 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f02f8017070 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.382+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02f8015070 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.383+0000 7f02eb400640 1 -- 172.21.15.6:0/2482375269 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f8005e50 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.383+0000 7f030f1e3740 10 client.25679 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.383+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55665a94b670 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.383+0000 7f02eb400640 1 -- 172.21.15.6:0/2482375269 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02f8043080 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f02eb400640 1 -- 172.21.15.6:0/2482375269 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02f8003ba0 con 0x55665aa23f10 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f030f1e3740 10 client.25679 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f030f1e3740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f030f1e3740 20 client.25679 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f030f1e3740 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f030f1e3740 4 client.25679 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:28:29.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55665a6c77b0 con 0x55665aa3d700 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.384+0000 7f0308e00640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.385+0000 7f02eb400640 10 client.25679 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.385+0000 7f0308e00640 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa97430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff79a097167d2e0 server_cookie=9c19db5c5abc29e7 in_seq=0 out_seq=0 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f02eb400640 1 -- 172.21.15.6:0/2482375269 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55665a6c77b0 con 0x55665aa3d700 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f02eb400640 10 client.25679 handle_command_reply: tid=0 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 client.25679 shutdown 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 msgr2=0x55665aa32d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa32d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f02f8003b70 tx=0x7f02f8004fd0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 shutdown_connections 2024-02-26T06:28:29.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55665aa3d700 0x55665aa97430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55665aa2d410 0x55665aa337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55665aa24830 0x55665aa33290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 --2- 172.21.15.6:0/2482375269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55665aa23f10 0x55665aa32d50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 >> 172.21.15.6:0/2482375269 conn(0x55665aa1e640 msgr2=0x55665aa1ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f02eb400640 0 client.25679 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.386+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 shutdown_connections 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.387+0000 7f030f1e3740 1 -- 172.21.15.6:0/2482375269 wait complete. 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.387+0000 7f030f1e3740 20 client.25679 trim_cache size 0 max 16384 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:28:29.388 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-02-26T06:28:29.389 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "f1a7401d-2b4c-4412-90d7-eddfff4fe8e0", 2024-02-26T06:28:29.389 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-02-26T06:28:29.389 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:28:29.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.388+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f02ec0a2340 msgr2=0x7f02ec0a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.388+0000 7f030c200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f02ec0a2340 0x7f02ec0a4800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005f40 tx=0x7f02fc009600 comp rx=0 tx=0).stop 2024-02-26T06:28:29.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.388+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 msgr2=0x7f030419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.388+0000 7f030c200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 0x7f030419fbb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000c00 tx=0x7f02f0004040 comp rx=0 tx=0).stop 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 shutdown_connections 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f02ec0a2340 0x7f02ec0a4800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0304199830 0x7f0304199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0304108660 0x7f030419fbb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 --2- 172.21.15.6:0/1616834792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0304103590 0x7f030419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 >> 172.21.15.6:0/1616834792 conn(0x7f03040fedd0 msgr2=0x7f0304101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 shutdown_connections 2024-02-26T06:28:29.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.389+0000 7f030c200640 1 -- 172.21.15.6:0/1616834792 wait complete. 2024-02-26T06:28:29.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:29.405 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:29.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 -- 172.21.15.6:0/96085130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41033e0 msgr2=0x7f18a41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 --2- 172.21.15.6:0/96085130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41033e0 0x7f18a41037c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1898009930 tx=0x7f189802bc20 comp rx=0 tx=0).stop 2024-02-26T06:28:29.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 -- 172.21.15.6:0/96085130 shutdown_connections 2024-02-26T06:28:29.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 --2- 172.21.15.6:0/96085130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41084b0 0x7f18a4110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 --2- 172.21.15.6:0/96085130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a4103d90 0x7f18a4107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 --2- 172.21.15.6:0/96085130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41033e0 0x7f18a41037c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 -- 172.21.15.6:0/96085130 >> 172.21.15.6:0/96085130 conn(0x7f18a40fec20 msgr2=0x7f18a4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 -- 172.21.15.6:0/96085130 shutdown_connections 2024-02-26T06:28:29.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.586+0000 7f18aa800640 1 -- 172.21.15.6:0/96085130 wait complete. 2024-02-26T06:28:29.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.587+0000 7f18aa800640 1 Processor -- start 2024-02-26T06:28:29.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 -- start start 2024-02-26T06:28:29.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 0x7f18a419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a41084b0 0x7f18a419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41996d0 0x7f18a4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a419a1a0 con 0x7f18a41084b0 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a419a310 con 0x7f18a41996d0 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18aa800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a419a4d0 con 0x7f18a41033e0 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.588+0000 7f18a9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 0x7f18a419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 0x7f18a419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58816/0 (socket says 172.21.15.6:58816) 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 -- 172.21.15.6:0/3120699616 learned_addr learned my addr 172.21.15.6:0/3120699616 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9e00640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41996d0 0x7f18a4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a8a00640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a41084b0 0x7f18a419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 -- 172.21.15.6:0/3120699616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41996d0 msgr2=0x7f18a4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41996d0 0x7f18a4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 -- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a41084b0 msgr2=0x7f18a419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a41084b0 0x7f18a419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 -- 172.21.15.6:0/3120699616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1898009560 con 0x7f18a41033e0 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a8a00640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a41084b0 0x7f18a419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.589+0000 7f18a9400640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 0x7f18a419f610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1898000c00 tx=0x7f189802cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.590+0000 7f18a2000640 1 -- 172.21.15.6:0/3120699616 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189802dea0 con 0x7f18a41033e0 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.590+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a41a6410 con 0x7f18a41033e0 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.590+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a41a6940 con 0x7f18a41033e0 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.590+0000 7f18a2000640 1 -- 172.21.15.6:0/3120699616 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1898004350 con 0x7f18a41033e0 2024-02-26T06:28:29.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.590+0000 7f18a2000640 1 -- 172.21.15.6:0/3120699616 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898003b20 con 0x7f18a41033e0 2024-02-26T06:28:29.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.595+0000 7f18aedc8740 1 Processor -- start 2024-02-26T06:28:29.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.595+0000 7f18a2000640 1 -- 172.21.15.6:0/3120699616 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f1898003c80 con 0x7f18a41033e0 2024-02-26T06:28:29.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.595+0000 7f18aedc8740 1 -- start start 2024-02-26T06:28:29.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a7d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18aedc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a734120 con 0x56087a7c6140 2024-02-26T06:28:29.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18aedc8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a731650 con 0x56087a7ce7d0 2024-02-26T06:28:29.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18aedc8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a72ce30 con 0x56087a7c5460 2024-02-26T06:28:29.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18a9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54246/0 (socket says 172.21.15.6:54246) 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.596+0000 7f18a9e00640 1 -- 172.21.15.6:0/2409927733 learned_addr learned my addr 172.21.15.6:0/2409927733 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a8a00640 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a7d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a9e00640 1 -- 172.21.15.6:0/2409927733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 msgr2=0x56087a7c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a9e00640 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a9e00640 1 -- 172.21.15.6:0/2409927733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 msgr2=0x56087a7d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a9e00640 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a7d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a9e00640 1 -- 172.21.15.6:0/2409927733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56087a6e6030 con 0x56087a7c6140 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.597+0000 7f18a8a00640 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a7d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.598+0000 7f18a2000640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f188c0a22c0 0x7f188c0a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.598+0000 7f18a9400640 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:29.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.598+0000 7f18a2000640 1 -- 172.21.15.6:0/3120699616 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f18980f0180 con 0x7f18a41033e0 2024-02-26T06:28:29.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.598+0000 7f18a9e00640 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7c4e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f189c009f50 tx=0x7f189c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd18d2eddd20988c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.598+0000 7f18a9400640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f188c0a22c0 0x7f188c0a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.599+0000 7f1896600640 1 -- 172.21.15.6:0/2409927733 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c004b20 con 0x56087a7c6140 2024-02-26T06:28:29.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.599+0000 7f1896600640 1 -- 172.21.15.6:0/2409927733 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f189c004c80 con 0x56087a7c6140 2024-02-26T06:28:29.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.599+0000 7f1896600640 1 -- 172.21.15.6:0/2409927733 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c005020 con 0x56087a7c6140 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.599+0000 7f18aedc8740 1 -- 172.21.15.6:0/2409927733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 msgr2=0x56087a7c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.599+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7c4e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f189c009f50 tx=0x7f189c0118d0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.599+0000 7f18a9400640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f188c0a22c0 0x7f188c0a4780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f18980ed900 tx=0x7f18980c3040 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 -- 172.21.15.6:0/2409927733 shutdown_connections 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a7d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2409927733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7c4e90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 -- 172.21.15.6:0/2409927733 >> 172.21.15.6:0/2409927733 conn(0x56087a78c570 msgr2=0x56087a7bfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 -- 172.21.15.6:0/2409927733 shutdown_connections 2024-02-26T06:28:29.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.600+0000 7f18aedc8740 1 -- 172.21.15.6:0/2409927733 wait complete. 2024-02-26T06:28:29.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.601+0000 7f18aedc8740 1 -- start start 2024-02-26T06:28:29.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.601+0000 7f18aedc8740 1 Processor -- start 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c5460 0x56087a7cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 0x56087a7cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7ce7d0 0x56087a7cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a6d1e90 con 0x56087a7ce7d0 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a6d1b30 con 0x56087a7c5460 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a49b1c0 con 0x56087a7c6140 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18aedc8740 10 client.0 _ll_register_callbacks cb 0x56087a6e5930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:29.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18a8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 0x56087a7cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18a8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 0x56087a7cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58836/0 (socket says 172.21.15.6:58836) 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c5460 0x56087a7cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c5460 0x56087a7cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48678/0 (socket says 172.21.15.6:48678) 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18a9e00640 1 -- 172.21.15.6:0/1774103197 learned_addr learned my addr 172.21.15.6:0/1774103197 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.602+0000 7f18a9e00640 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7ce7d0 0x56087a7cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f18a8a00640 1 -- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c5460 msgr2=0x56087a7cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f18a8a00640 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c5460 0x56087a7cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f18a8a00640 1 -- 172.21.15.6:0/1774103197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7ce7d0 msgr2=0x56087a7cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f18a8a00640 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7ce7d0 0x56087a7cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f18a8a00640 1 -- 172.21.15.6:0/1774103197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c009c00 con 0x56087a7c6140 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f18a8a00640 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 0x56087a7cbbf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f18a419af20 tx=0x7f18900117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f1a3310c5ea5fa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.603+0000 7f188aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f188aa00640 1 -- 172.21.15.6:0/1774103197 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890004220 con 0x56087a7c6140 2024-02-26T06:28:29.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f188aa00640 1 -- 172.21.15.6:0/1774103197 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f1890004380 con 0x56087a7c6140 2024-02-26T06:28:29.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56087a42c7e0 con 0x56087a7c6140 2024-02-26T06:28:29.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f188aa00640 1 -- 172.21.15.6:0/1774103197 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1890005020 con 0x56087a7c6140 2024-02-26T06:28:29.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f18aedc8740 10 client.25672 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56087a72b4b0 con 0x56087a7c6140 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f188aa00640 1 -- 172.21.15.6:0/1774103197 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56087a42c7e0 con 0x56087a7c6140 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.604+0000 7f188aa00640 1 -- 172.21.15.6:0/1774103197 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f189000c070 con 0x56087a7c6140 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18aedc8740 10 client.25672 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18aedc8740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18aedc8740 20 client.25672 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18aedc8740 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a7dea90 0x56087a7e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18aedc8740 4 client.25672 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56087a469720 con 0x56087a7dea90 2024-02-26T06:28:29.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.605+0000 7f18a9e00640 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a7dea90 0x56087a7e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.606+0000 7f18a9e00640 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a7dea90 0x56087a7e0eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c249f285007d813a server_cookie=38e2d8aa86729e6f in_seq=0 out_seq=0 2024-02-26T06:28:29.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.606+0000 7f188aa00640 10 client.25672 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.607+0000 7f188aa00640 1 -- 172.21.15.6:0/1774103197 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56087a469720 con 0x56087a7dea90 2024-02-26T06:28:29.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.607+0000 7f188aa00640 10 client.25672 handle_command_reply: tid=0 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.607+0000 7f18aedc8740 1 client.25672 shutdown 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.607+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 msgr2=0x56087a7cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.607+0000 7f18aedc8740 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 0x56087a7cbbf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f18a419af20 tx=0x7f18900117b0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 shutdown_connections 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a7dea90 0x56087a7e0eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7ce7d0 0x56087a7cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c6140 0x56087a7cbbf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 --2- 172.21.15.6:0/1774103197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c5460 0x56087a7cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f188aa00640 0 client.25672 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 >> 172.21.15.6:0/1774103197 conn(0x56087a78c570 msgr2=0x56087a7c1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 shutdown_connections 2024-02-26T06:28:29.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 1 -- 172.21.15.6:0/1774103197 wait complete. 2024-02-26T06:28:29.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.608+0000 7f18aedc8740 20 client.25672 trim_cache size 0 max 16384 2024-02-26T06:28:29.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 Processor -- start 2024-02-26T06:28:29.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 -- start start 2024-02-26T06:28:29.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 0x56087a7cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c6140 0x56087a7cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7ce7d0 0x56087a7c8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a7c46e0 con 0x56087a7c5460 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a7cc7c0 con 0x56087a7c6140 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.639+0000 7f18aedc8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a7c6520 con 0x56087a7ce7d0 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 0x56087a7cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 0x56087a7cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54260/0 (socket says 172.21.15.6:54260) 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a9e00640 1 -- 172.21.15.6:0/2668874593 learned_addr learned my addr 172.21.15.6:0/2668874593 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a8a00640 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c6140 0x56087a7cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a9e00640 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7ce7d0 0x56087a7c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a8a00640 1 -- 172.21.15.6:0/2668874593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7ce7d0 msgr2=0x56087a7c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.640+0000 7f18a8a00640 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7ce7d0 0x56087a7c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.641+0000 7f18a8a00640 1 -- 172.21.15.6:0/2668874593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 msgr2=0x56087a7cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.641+0000 7f18a8a00640 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 0x56087a7cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.641+0000 7f18a8a00640 1 -- 172.21.15.6:0/2668874593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56087a6c7920 con 0x56087a7c6140 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.641+0000 7f18a9e00640 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 0x56087a7cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:29.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.641+0000 7f18a8a00640 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c6140 0x56087a7cc180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1890004f30 tx=0x7f1890011f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac78a547979f64b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.641+0000 7f1895c00640 1 -- 172.21.15.6:0/2668874593 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890009ac0 con 0x56087a7c6140 2024-02-26T06:28:29.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.642+0000 7f1895c00640 1 -- 172.21.15.6:0/2668874593 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1890011cb0 con 0x56087a7c6140 2024-02-26T06:28:29.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.642+0000 7f1895c00640 1 -- 172.21.15.6:0/2668874593 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890012a00 con 0x56087a7c6140 2024-02-26T06:28:29.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.642+0000 7f18aedc8740 1 -- 172.21.15.6:0/2668874593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c6140 msgr2=0x56087a7cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c6140 0x56087a7cc180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1890004f30 tx=0x7f1890011f50 comp rx=0 tx=0).stop 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 -- 172.21.15.6:0/2668874593 shutdown_connections 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7ce7d0 0x56087a7c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7c6140 0x56087a7cc180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 --2- 172.21.15.6:0/2668874593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c5460 0x56087a7cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 -- 172.21.15.6:0/2668874593 >> 172.21.15.6:0/2668874593 conn(0x56087a78c570 msgr2=0x56087a7c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 -- 172.21.15.6:0/2668874593 shutdown_connections 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.643+0000 7f18aedc8740 1 -- 172.21.15.6:0/2668874593 wait complete. 2024-02-26T06:28:29.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 -- start start 2024-02-26T06:28:29.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 Processor -- start 2024-02-26T06:28:29.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7d4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a834f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a7d54e0 con 0x56087a7c6140 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a8354a0 con 0x56087a7ce7d0 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.644+0000 7f18aedc8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087a835610 con 0x56087a7c5460 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18aedc8740 10 client.0 _ll_register_callbacks cb 0x7f18900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a8a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7d4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7d48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:58846/0 (socket says 172.21.15.6:58846) 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a9e00640 1 -- 172.21.15.6:0/350960388 learned_addr learned my addr 172.21.15.6:0/350960388 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a9e00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a834f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a8a00640 1 -- 172.21.15.6:0/350960388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 msgr2=0x56087a7d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a8a00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7d48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a8a00640 1 -- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 msgr2=0x56087a834f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a8a00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a834f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a8a00640 1 -- 172.21.15.6:0/350960388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c035040 con 0x56087a7c6140 2024-02-26T06:28:29.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.645+0000 7f18a9e00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a834f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.646+0000 7f18a8a00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7d4de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x56087a7c70b0 tx=0x7f1890011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f28792937cd8aa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.646+0000 7f188aa00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.646+0000 7f188aa00640 1 -- 172.21.15.6:0/350960388 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18900056f0 con 0x56087a7c6140 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.646+0000 7f188aa00640 1 -- 172.21.15.6:0/350960388 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f1890009420 con 0x56087a7c6140 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.646+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56087a4b8890 con 0x56087a7c6140 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f188aa00640 1 -- 172.21.15.6:0/350960388 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18900051e0 con 0x56087a7c6140 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 10 client.15558 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56087a72b4b0 con 0x56087a7c6140 2024-02-26T06:28:29.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f188aa00640 1 -- 172.21.15.6:0/350960388 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1890040080 con 0x56087a7c6140 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f188aa00640 1 -- 172.21.15.6:0/350960388 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f189000c070 con 0x56087a7c6140 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 20 client.15558 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a836a50 0x56087a838e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 4 client.15558 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.647+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56087a3d8700 con 0x56087a836a50 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.648+0000 7f18a9e00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a836a50 0x56087a838e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:29.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.648+0000 7f188aa00640 10 client.15558 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.648+0000 7f18a9e00640 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a836a50 0x56087a838e70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b1c25a93da26f server_cookie=7f7126ffbdac10f8 in_seq=0 out_seq=0 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f188aa00640 1 -- 172.21.15.6:0/350960388 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56087a3d8700 con 0x56087a836a50 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f188aa00640 10 client.15558 handle_command_reply: tid=0 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 client.15558 shutdown 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 msgr2=0x56087a7d4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7d4de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x56087a7c70b0 tx=0x7f1890011cb0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 shutdown_connections 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56087a836a50 0x56087a838e70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56087a7ce7d0 0x56087a834f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56087a7c6140 0x56087a7d4de0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 --2- 172.21.15.6:0/350960388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56087a7c5460 0x56087a7d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 >> 172.21.15.6:0/350960388 conn(0x56087a78c570 msgr2=0x56087a7c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.649+0000 7f188aa00640 0 client.15558 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.650+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 shutdown_connections 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.650+0000 7f18aedc8740 1 -- 172.21.15.6:0/350960388 wait complete. 2024-02-26T06:28:29.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.650+0000 7f18aedc8740 20 client.15558 trim_cache size 0 max 16384 2024-02-26T06:28:29.651 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:28:29.651 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T06:28:29.651 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-02-26T06:28:29.651 INFO:teuthology.orchestra.run.smithi006.stdout: "f1a7401d-2b4c-4412-90d7-eddfff4fe8e0": { 2024-02-26T06:28:29.651 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-02-26T06:28:29.651 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "f1a7401d-2b4c-4412-90d7-eddfff4fe8e0", 2024-02-26T06:28:29.652 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-02-26T06:28:29.652 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:28:29.652 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:28:29.652 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f188c0a22c0 msgr2=0x7f188c0a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f188c0a22c0 0x7f188c0a4780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f18980ed900 tx=0x7f18980c3040 comp rx=0 tx=0).stop 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 msgr2=0x7f18a419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 0x7f18a419f610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1898000c00 tx=0x7f189802cd70 comp rx=0 tx=0).stop 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 shutdown_connections 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f188c0a22c0 0x7f188c0a4780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18a41996d0 0x7f18a4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18a41084b0 0x7f18a419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 --2- 172.21.15.6:0/3120699616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a41033e0 0x7f18a419f610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 >> 172.21.15.6:0/3120699616 conn(0x7f18a40fec20 msgr2=0x7f18a40fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:29.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 shutdown_connections 2024-02-26T06:28:29.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:29.652+0000 7f18aa800640 1 -- 172.21.15.6:0/3120699616 wait complete. 2024-02-26T06:28:29.676 INFO:tasks.cephfs.filesystem:scrub status for tag:f1a7401d-2b4c-4412-90d7-eddfff4fe8e0 - {'path': '/', 'tag': 'f1a7401d-2b4c-4412-90d7-eddfff4fe8e0', 'options': 'recursive,force'} 2024-02-26T06:28:30.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:29 smithi006 ceph-mon[31790]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-02-26T06:28:30.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:29 smithi167 ceph-mon[36229]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-02-26T06:28:30.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:29 smithi090 ceph-mon[35681]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-02-26T06:28:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:30 smithi006 ceph-mon[31790]: scrub queued for path: / 2024-02-26T06:28:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:30 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:30 smithi006 ceph-mon[31790]: scrub summary: active paths [/] 2024-02-26T06:28:31.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:30 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:31.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:30 smithi167 ceph-mon[36229]: scrub queued for path: / 2024-02-26T06:28:31.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:30 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:30 smithi167 ceph-mon[36229]: scrub summary: active paths [/] 2024-02-26T06:28:31.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:30 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:30 smithi090 ceph-mon[35681]: scrub queued for path: / 2024-02-26T06:28:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:30 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:30 smithi090 ceph-mon[35681]: scrub summary: active paths [/] 2024-02-26T06:28:31.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:30 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:28:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:31 smithi006 ceph-mon[31790]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-02-26T06:28:32.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:31 smithi006 ceph-mon[31790]: scrub summary: idle 2024-02-26T06:28:32.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:31 smithi167 ceph-mon[36229]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-02-26T06:28:32.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:31 smithi167 ceph-mon[36229]: scrub summary: idle 2024-02-26T06:28:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:31 smithi090 ceph-mon[35681]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-02-26T06:28:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:31 smithi090 ceph-mon[35681]: scrub summary: idle 2024-02-26T06:28:34.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:33 smithi006 ceph-mon[31790]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-26T06:28:34.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:33 smithi167 ceph-mon[36229]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-26T06:28:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:33 smithi090 ceph-mon[35681]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-26T06:28:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 -- 172.21.15.6:0/676039716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81062a0 msgr2=0x7f87b810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 --2- 172.21.15.6:0/676039716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81062a0 0x7f87b810de70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009cd0 tx=0x7f87b002c0e0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 -- 172.21.15.6:0/676039716 shutdown_connections 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 --2- 172.21.15.6:0/676039716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81062a0 0x7f87b810de70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 --2- 172.21.15.6:0/676039716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b8101b80 0x7f87b8105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 --2- 172.21.15.6:0/676039716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b81011d0 0x7f87b81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 -- 172.21.15.6:0/676039716 >> 172.21.15.6:0/676039716 conn(0x7f87b80fca90 msgr2=0x7f87b80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 -- 172.21.15.6:0/676039716 shutdown_connections 2024-02-26T06:28:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.630+0000 7f87bd400640 1 -- 172.21.15.6:0/676039716 wait complete. 2024-02-26T06:28:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.631+0000 7f87bd400640 1 Processor -- start 2024-02-26T06:28:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 -- start start 2024-02-26T06:28:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81011d0 0x7f87b819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b81062a0 0x7f87b819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b8199660 0x7f87b8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b819a050 con 0x7f87b81062a0 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b819a1c0 con 0x7f87b81011d0 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.632+0000 7f87bd400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b819a380 con 0x7f87b8199660 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b81062a0 0x7f87b819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81011d0 0x7f87b819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b81062a0 0x7f87b819f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54290/0 (socket says 172.21.15.6:54290) 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7400640 1 -- 172.21.15.6:0/892249687 learned_addr learned my addr 172.21.15.6:0/892249687 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87bca00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b8199660 0x7f87b8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.634 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 -- 172.21.15.6:0/892249687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b8199660 msgr2=0x7f87b8199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.634 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b8199660 0x7f87b8199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.634 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 -- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b81062a0 msgr2=0x7f87b819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.634 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b81062a0 0x7f87b819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 -- 172.21.15.6:0/892249687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009900 con 0x7f87b81011d0 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.633+0000 7f87b7e00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81011d0 0x7f87b819f460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.634+0000 7f87b4c00640 1 -- 172.21.15.6:0/892249687 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800ccc0 con 0x7f87b81011d0 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.634+0000 7f87b4c00640 1 -- 172.21.15.6:0/892249687 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87a800ce20 con 0x7f87b81011d0 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.634+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b81a6320 con 0x7f87b81011d0 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.634+0000 7f87b4c00640 1 -- 172.21.15.6:0/892249687 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8010430 con 0x7f87b81011d0 2024-02-26T06:28:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.634+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b81a6830 con 0x7f87b81011d0 2024-02-26T06:28:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.638+0000 7f87c1816740 1 Processor -- start 2024-02-26T06:28:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.638+0000 7f87c1816740 1 -- start start 2024-02-26T06:28:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87b4c00640 1 -- 172.21.15.6:0/892249687 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f87a80046c0 con 0x7f87b81011d0 2024-02-26T06:28:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87c1816740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538153fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87c1816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x565381540440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87c1816740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x565381554140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87c1816740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653814fb5f0 con 0x565381551d50 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87c1816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653814ad890 con 0x56538153ffc0 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87c1816740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653814aadc0 con 0x565381540d30 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87b7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x565381554140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87b7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x565381554140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54306/0 (socket says 172.21.15.6:54306) 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87b7400640 1 -- 172.21.15.6:0/414798188 learned_addr learned my addr 172.21.15.6:0/414798188 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.639+0000 7f87bca00640 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538153fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7e00640 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x565381540440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7400640 1 -- 172.21.15.6:0/414798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 msgr2=0x56538153fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7400640 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538153fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7400640 1 -- 172.21.15.6:0/414798188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 msgr2=0x565381540440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7400640 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x565381540440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7400640 1 -- 172.21.15.6:0/414798188 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565381427ad0 con 0x565381551d50 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.640+0000 7f87b7400640 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x565381554140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009c40 tx=0x7f87a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9ad561ddf43cfe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.641+0000 7f87b4c00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f879c0a23a0 0x7f879c0a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.641+0000 7f87ad200640 1 -- 172.21.15.6:0/414798188 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4004230 con 0x565381551d50 2024-02-26T06:28:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.641+0000 7f87b4c00640 1 -- 172.21.15.6:0/892249687 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f87a8014070 con 0x7f87b81011d0 2024-02-26T06:28:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.641+0000 7f87ad200640 1 -- 172.21.15.6:0/414798188 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87a4004390 con 0x565381551d50 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.641+0000 7f87ad200640 1 -- 172.21.15.6:0/414798188 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4004d80 con 0x565381551d50 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.641+0000 7f87bca00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f879c0a23a0 0x7f879c0a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 -- 172.21.15.6:0/414798188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 msgr2=0x565381554140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x565381554140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009c40 tx=0x7f87a4011810 comp rx=0 tx=0).stop 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87bca00640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f879c0a23a0 0x7f879c0a4860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005cc0 tx=0x7f87b0005c50 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 -- 172.21.15.6:0/414798188 shutdown_connections 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x565381554140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x565381540440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 --2- 172.21.15.6:0/414798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538153fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.642+0000 7f87c1816740 1 -- 172.21.15.6:0/414798188 >> 172.21.15.6:0/414798188 conn(0x565381507480 msgr2=0x56538153acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.643+0000 7f87c1816740 1 -- 172.21.15.6:0/414798188 shutdown_connections 2024-02-26T06:28:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.643+0000 7f87c1816740 1 -- 172.21.15.6:0/414798188 wait complete. 2024-02-26T06:28:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.643+0000 7f87c1816740 1 -- start start 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.643+0000 7f87c1816740 1 Processor -- start 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x56538154ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538154f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x56538154f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653814a65a0 con 0x565381551d50 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538144bb20 con 0x56538153ffc0 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538144b7c0 con 0x565381540d30 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87c1816740 10 client.0 _ll_register_callbacks cb 0x56538145f520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x56538154ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x56538154ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48738/0 (socket says 172.21.15.6:48738) 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87b7e00640 1 -- 172.21.15.6:0/4152380278 learned_addr learned my addr 172.21.15.6:0/4152380278 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87bca00640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x56538154f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.644+0000 7f87b7400640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538154f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87b7e00640 1 -- 172.21.15.6:0/4152380278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 msgr2=0x56538154f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87b7e00640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538154f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87b7e00640 1 -- 172.21.15.6:0/4152380278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 msgr2=0x56538154f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87b7e00640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x56538154f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87b7e00640 1 -- 172.21.15.6:0/4152380278 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a40098a0 con 0x56538153ffc0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87bca00640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x56538154f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87b7e00640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x56538154ed70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c6a0 tx=0x7f87a80d1f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e7077f8fa1c1c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87a1600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87a1600640 1 -- 172.21.15.6:0/4152380278 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80a3280 con 0x56538153ffc0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87a1600640 1 -- 172.21.15.6:0/4152380278 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f87a809b470 con 0x56538153ffc0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565381478ca0 con 0x56538153ffc0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87a1600640 1 -- 172.21.15.6:0/4152380278 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a809a8b0 con 0x56538153ffc0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87c1816740 10 client.25709 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.645+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5653814a4c20 con 0x56538153ffc0 2024-02-26T06:28:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87a1600640 1 -- 172.21.15.6:0/4152380278 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565381478ca0 con 0x56538153ffc0 2024-02-26T06:28:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87a1600640 1 -- 172.21.15.6:0/4152380278 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87a8085050 con 0x56538153ffc0 2024-02-26T06:28:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87c1816740 10 client.25709 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87c1816740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87c1816740 20 client.25709 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87c1816740 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565381561ce0 0x5653815640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87c1816740 4 client.25709 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56538149ccb0 con 0x565381561ce0 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.646+0000 7f87b7400640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565381561ce0 0x5653815640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.647+0000 7f87b7400640 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565381561ce0 0x5653815640d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff4db7669166c5e server_cookie=c034fc38ac65277a in_seq=0 out_seq=0 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.647+0000 7f87a1600640 10 client.25709 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87a1600640 1 -- 172.21.15.6:0/4152380278 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56538149ccb0 con 0x565381561ce0 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87a1600640 10 client.25709 handle_command_reply: tid=0 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 client.25709 shutdown 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 msgr2=0x56538154ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x56538154ed70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c6a0 tx=0x7f87a80d1f60 comp rx=0 tx=0).stop 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 shutdown_connections 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565381561ce0 0x5653815640d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565381551d50 0x56538154f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x56538154f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 --2- 172.21.15.6:0/4152380278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56538153ffc0 0x56538154ed70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 >> 172.21.15.6:0/4152380278 conn(0x565381507480 msgr2=0x56538153ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87a1600640 0 client.25709 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.648+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 shutdown_connections 2024-02-26T06:28:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.649+0000 7f87c1816740 1 -- 172.21.15.6:0/4152380278 wait complete. 2024-02-26T06:28:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.649+0000 7f87c1816740 20 client.25709 trim_cache size 0 max 16384 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 Processor -- start 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 -- start start 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x56538153a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381540d30 0x56538153ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381551d50 0x56538154d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653811eee60 con 0x56538153ffc0 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538153bcb0 con 0x565381540d30 2024-02-26T06:28:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87c1816740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x565381561ae0 con 0x565381551d50 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x56538153a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x56538153a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54328/0 (socket says 172.21.15.6:54328) 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87b7e00640 1 -- 172.21.15.6:0/1170924889 learned_addr learned my addr 172.21.15.6:0/1170924889 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.663+0000 7f87bca00640 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381551d50 0x56538154d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7400640 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381540d30 0x56538153ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7e00640 1 -- 172.21.15.6:0/1170924889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381551d50 msgr2=0x56538154d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7e00640 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381551d50 0x56538154d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7e00640 1 -- 172.21.15.6:0/1170924889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381540d30 msgr2=0x56538153ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7e00640 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381540d30 0x56538153ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7e00640 1 -- 172.21.15.6:0/1170924889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565381441500 con 0x56538153ffc0 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87b7e00640 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x56538153a7e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f87a80971d0 tx=0x7f87a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a29dc647be285a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87a3e00640 1 -- 172.21.15.6:0/1170924889 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80b92e0 con 0x56538153ffc0 2024-02-26T06:28:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.664+0000 7f87a3e00640 1 -- 172.21.15.6:0/1170924889 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87a809e020 con 0x56538153ffc0 2024-02-26T06:28:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.665+0000 7f87a3e00640 1 -- 172.21.15.6:0/1170924889 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80a32c0 con 0x56538153ffc0 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 -- 172.21.15.6:0/1170924889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 msgr2=0x56538153a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x56538153a7e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f87a80971d0 tx=0x7f87a800cf90 comp rx=0 tx=0).stop 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 -- 172.21.15.6:0/1170924889 shutdown_connections 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381551d50 0x56538154d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381540d30 0x56538153ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 --2- 172.21.15.6:0/1170924889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x56538153a7e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 -- 172.21.15.6:0/1170924889 >> 172.21.15.6:0/1170924889 conn(0x565381507480 msgr2=0x56538154a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 -- 172.21.15.6:0/1170924889 shutdown_connections 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.666+0000 7f87c1816740 1 -- 172.21.15.6:0/1170924889 wait complete. 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 -- start start 2024-02-26T06:28:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 Processor -- start 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x565381556b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x5653815570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381551d50 0x5653815575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653815b7240 con 0x56538153ffc0 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653815b73b0 con 0x565381551d50 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.667+0000 7f87c1816740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653815b7570 con 0x565381540d30 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87c1816740 10 client.0 _ll_register_callbacks cb 0x7f87a80d1a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x565381556b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x565381556b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54342/0 (socket says 172.21.15.6:54342) 2024-02-26T06:28:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 -- 172.21.15.6:0/4313264 learned_addr learned my addr 172.21.15.6:0/4313264 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7400640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x5653815570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87bca00640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381551d50 0x5653815575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 -- 172.21.15.6:0/4313264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 msgr2=0x5653815570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x5653815570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 -- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381551d50 msgr2=0x5653815575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381551d50 0x5653815575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87b7e00640 1 -- 172.21.15.6:0/4313264 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653814a4c20 con 0x56538153ffc0 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.668+0000 7f87bca00640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381551d50 0x5653815575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.669+0000 7f87b7e00640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x565381556b60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f87a809aa70 tx=0x7f87a809a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a88755a5f808eb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.669+0000 7f87a1600640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.669+0000 7f87a1600640 1 -- 172.21.15.6:0/4313264 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80d1b40 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.669+0000 7f87a1600640 1 -- 172.21.15.6:0/4313264 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f87a809ad70 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.669+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653814cbd00 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87a1600640 1 -- 172.21.15.6:0/4313264 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a80b92e0 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565381467290 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87a1600640 1 -- 172.21.15.6:0/4313264 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5653814cbd00 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87a1600640 1 -- 172.21.15.6:0/4313264 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87a8085040 con 0x56538153ffc0 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 20 client.15576 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5653815b8810 0x5653815bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 4 client.15576 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653811a6600 con 0x5653815b8810 2024-02-26T06:28:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.670+0000 7f87b7400640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5653815b8810 0x5653815bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.671+0000 7f87b7400640 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5653815b8810 0x5653815bac00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2736b4f6d46b1e46 server_cookie=bf645b8e2ddf3434 in_seq=0 out_seq=0 2024-02-26T06:28:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.671+0000 7f87a1600640 10 client.15576 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87a1600640 1 -- 172.21.15.6:0/4313264 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653811a6600 con 0x5653815b8810 2024-02-26T06:28:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87a1600640 10 client.15576 handle_command_reply: tid=0 2024-02-26T06:28:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 client.15576 shutdown 2024-02-26T06:28:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 msgr2=0x565381556b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x565381556b60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f87a809aa70 tx=0x7f87a809a400 comp rx=0 tx=0).stop 2024-02-26T06:28:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 shutdown_connections 2024-02-26T06:28:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5653815b8810 0x5653815bac00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565381551d50 0x5653815575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565381540d30 0x5653815570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 --2- 172.21.15.6:0/4313264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56538153ffc0 0x565381556b60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 >> 172.21.15.6:0/4313264 conn(0x565381507480 msgr2=0x56538153a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87a1600640 0 client.15576 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 shutdown_connections 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.675+0000 7f87c1816740 1 -- 172.21.15.6:0/4313264 wait complete. 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87c1816740 20 client.15576 trim_cache size 0 max 16384 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:28:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f879c0a23a0 msgr2=0x7f879c0a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f879c0a23a0 0x7f879c0a4860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005cc0 tx=0x7f87b0005c50 comp rx=0 tx=0).stop 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81011d0 msgr2=0x7f87b819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81011d0 0x7f87b819f460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 shutdown_connections 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f879c0a23a0 0x7f879c0a4860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b8199660 0x7f87b8199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87b81062a0 0x7f87b819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 --2- 172.21.15.6:0/892249687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b81011d0 0x7f87b819f460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.676+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 >> 172.21.15.6:0/892249687 conn(0x7f87b80fca90 msgr2=0x7f87b80fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.677+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 shutdown_connections 2024-02-26T06:28:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:35.677+0000 7f87bd400640 1 -- 172.21.15.6:0/892249687 wait complete. 2024-02-26T06:28:36.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:35 smithi006 ceph-mon[31790]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-02-26T06:28:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:35 smithi167 ceph-mon[36229]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-02-26T06:28:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:35 smithi090 ceph-mon[35681]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-02-26T06:28:38.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:37 smithi006 ceph-mon[31790]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-26T06:28:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:37 smithi167 ceph-mon[36229]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-26T06:28:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:37 smithi090 ceph-mon[35681]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-26T06:28:40.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:39 smithi006 ceph-mon[31790]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 18 op/s 2024-02-26T06:28:40.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:39 smithi167 ceph-mon[36229]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 18 op/s 2024-02-26T06:28:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:39 smithi090 ceph-mon[35681]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 18 op/s 2024-02-26T06:28:42.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:41 smithi090 ceph-mon[35681]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-26T06:28:42.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:41 smithi006 ceph-mon[31790]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-26T06:28:42.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:41 smithi167 ceph-mon[36229]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-26T06:28:44.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:43 smithi006 ceph-mon[31790]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-26T06:28:44.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:43 smithi167 ceph-mon[36229]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-26T06:28:44.360 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:43 smithi090 ceph-mon[35681]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-26T06:28:46.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:45 smithi006 ceph-mon[31790]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-26T06:28:46.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:45 smithi167 ceph-mon[36229]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-26T06:28:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:45 smithi090 ceph-mon[35681]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-26T06:28:48.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:47 smithi006 ceph-mon[31790]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-26T06:28:48.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:47 smithi167 ceph-mon[36229]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-26T06:28:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:47 smithi090 ceph-mon[35681]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.812+0000 7f68cf000640 1 -- 172.21.15.6:0/3270057254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c810d820 msgr2=0x7f68c810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.812+0000 7f68cf000640 1 --2- 172.21.15.6:0/3270057254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c810d820 0x7f68c810fc10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009be0 tx=0x7f68c002bfa0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 -- 172.21.15.6:0/3270057254 shutdown_connections 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 --2- 172.21.15.6:0/3270057254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c810d820 0x7f68c810fc10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 --2- 172.21.15.6:0/3270057254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68c8073e50 0x7f68c810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 --2- 172.21.15.6:0/3270057254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073530 0x7f68c8073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 -- 172.21.15.6:0/3270057254 >> 172.21.15.6:0/3270057254 conn(0x7f68c80fca90 msgr2=0x7f68c80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 -- 172.21.15.6:0/3270057254 shutdown_connections 2024-02-26T06:28:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.813+0000 7f68cf000640 1 -- 172.21.15.6:0/3270057254 wait complete. 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 Processor -- start 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 -- start start 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 0x7f68c819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073e50 0x7f68c819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68c810d820 0x7f68c8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8199c70 con 0x7f68c810d820 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8199de0 con 0x7f68c8073e50 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.814+0000 7f68cf000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8199fa0 con 0x7f68c8073530 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cdc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 0x7f68c819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cdc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 0x7f68c819f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49964/0 (socket says 172.21.15.6:49964) 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cdc00640 1 -- 172.21.15.6:0/1488672313 learned_addr learned my addr 172.21.15.6:0/1488672313 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cd200640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073e50 0x7f68c819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68ce600640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68c810d820 0x7f68c8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cd200640 1 -- 172.21.15.6:0/1488672313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 msgr2=0x7f68c819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cd200640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 0x7f68c819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cd200640 1 -- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68c810d820 msgr2=0x7f68c8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cd200640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68c810d820 0x7f68c8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cd200640 1 -- 172.21.15.6:0/1488672313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009840 con 0x7f68c8073e50 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.815+0000 7f68cdc00640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 0x7f68c819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.816+0000 7f68cd200640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073e50 0x7f68c819f960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f68b800e790 tx=0x7f68b800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.816+0000 7f68c6000640 1 -- 172.21.15.6:0/1488672313 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800ccc0 con 0x7f68c8073e50 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.816+0000 7f68c6000640 1 -- 172.21.15.6:0/1488672313 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68b800eea0 con 0x7f68c8073e50 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.816+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c819a280 con 0x7f68c8073e50 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.816+0000 7f68c6000640 1 -- 172.21.15.6:0/1488672313 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8018730 con 0x7f68c8073e50 2024-02-26T06:28:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.816+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c81a64c0 con 0x7f68c8073e50 2024-02-26T06:28:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.820+0000 7f68c6000640 1 -- 172.21.15.6:0/1488672313 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f68b8010430 con 0x7f68c8073e50 2024-02-26T06:28:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 Processor -- start 2024-02-26T06:28:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 -- start start 2024-02-26T06:28:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c5d30 0x561cb90c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c4fc0 0x561cb90c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90ce540 0x561cb90d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb90805f0 con 0x561cb90ce540 2024-02-26T06:28:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb9032890 con 0x561cb90c4fc0 2024-02-26T06:28:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.821+0000 7f68d35a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb902fdc0 con 0x561cb90c5d30 2024-02-26T06:28:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68ce600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c5d30 0x561cb90c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68ce600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c5d30 0x561cb90c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:49970/0 (socket says 172.21.15.6:49970) 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68ce600640 1 -- 172.21.15.6:0/151885398 learned_addr learned my addr 172.21.15.6:0/151885398 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cd200640 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90ce540 0x561cb90d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68c6000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f68b00a22a0 0x7f68b00a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cdc00640 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c4fc0 0x561cb90c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68c6000640 1 -- 172.21.15.6:0/1488672313 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f68b8014070 con 0x7f68c8073e50 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68ce600640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f68b00a22a0 0x7f68b00a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cdc00640 1 -- 172.21.15.6:0/151885398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c5d30 msgr2=0x561cb90c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cdc00640 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c5d30 0x561cb90c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cdc00640 1 -- 172.21.15.6:0/151885398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90ce540 msgr2=0x561cb90d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cdc00640 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90ce540 0x561cb90d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cdc00640 1 -- 172.21.15.6:0/151885398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb8facad0 con 0x561cb90c4fc0 2024-02-26T06:28:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.822+0000 7f68cd200640 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90ce540 0x561cb90d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.823+0000 7f68cdc00640 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c4fc0 0x561cb90c5440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0114f0 tx=0x7f68bc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c289705f903eb224 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.823+0000 7f68b6a00640 1 -- 172.21.15.6:0/151885398 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc004030 con 0x561cb90c4fc0 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.823+0000 7f68b6a00640 1 -- 172.21.15.6:0/151885398 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68bc0035f0 con 0x561cb90c4fc0 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.823+0000 7f68b6a00640 1 -- 172.21.15.6:0/151885398 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00f930 con 0x561cb90c4fc0 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 -- 172.21.15.6:0/151885398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c4fc0 msgr2=0x561cb90c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c4fc0 0x561cb90c5440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0114f0 tx=0x7f68bc0119b0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68ce600640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f68b00a22a0 0x7f68b00a4760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68c002cfd0 tx=0x7f68c002bfa0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 -- 172.21.15.6:0/151885398 shutdown_connections 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90ce540 0x561cb90d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c4fc0 0x561cb90c5440 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 --2- 172.21.15.6:0/151885398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c5d30 0x561cb90c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 -- 172.21.15.6:0/151885398 >> 172.21.15.6:0/151885398 conn(0x561cb908c480 msgr2=0x561cb90bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 -- 172.21.15.6:0/151885398 shutdown_connections 2024-02-26T06:28:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.824+0000 7f68d35a0740 1 -- 172.21.15.6:0/151885398 wait complete. 2024-02-26T06:28:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.825+0000 7f68d35a0740 1 -- start start 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.825+0000 7f68d35a0740 1 Processor -- start 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb902b5a0 con 0x561cb90c4fc0 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb8fd0b20 con 0x561cb90c5d30 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb8fd07c0 con 0x561cb90ce540 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68d35a0740 10 client.0 _ll_register_callbacks cb 0x561cb8fe4520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68cdc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68cdc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37100/0 (socket says 172.21.15.6:37100) 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68cdc00640 1 -- 172.21.15.6:0/1152034001 learned_addr learned my addr 172.21.15.6:0/1152034001 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68cd200640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.826+0000 7f68ce600640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cd200640 1 -- 172.21.15.6:0/1152034001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 msgr2=0x561cb90cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cd200640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cd200640 1 -- 172.21.15.6:0/1152034001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 msgr2=0x561cb90cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cd200640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cd200640 1 -- 172.21.15.6:0/1152034001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc009cd0 con 0x561cb90c5d30 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68ce600640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cdc00640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68cd200640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f68b80cba10 tx=0x7f68b809a700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4e39c84c2e6142f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68ab400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68ab400640 1 -- 172.21.15.6:0/1152034001 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80a33a0 con 0x561cb90c5d30 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68ab400640 1 -- 172.21.15.6:0/1152034001 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f68b80cbcc0 con 0x561cb90c5d30 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cb8ffdca0 con 0x561cb90c5d30 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.827+0000 7f68ab400640 1 -- 172.21.15.6:0/1152034001 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68b80cbe80 con 0x561cb90c5d30 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 10 client.25727 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561cb9029c20 con 0x561cb90c5d30 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68ab400640 1 -- 172.21.15.6:0/1152034001 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561cb8ffdca0 con 0x561cb90c5d30 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68ab400640 1 -- 172.21.15.6:0/1152034001 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f68b8096a90 con 0x561cb90c5d30 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 10 client.25727 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 20 client.25727 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb90de540 0x561cb90e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 4 client.25727 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.828+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cb9021cb0 con 0x561cb90de540 2024-02-26T06:28:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.829+0000 7f68cdc00640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb90de540 0x561cb90e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.829+0000 7f68cdc00640 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb90de540 0x561cb90e0960 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9008df092cb472e3 server_cookie=856e61560db97945 in_seq=0 out_seq=0 2024-02-26T06:28:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.829+0000 7f68ab400640 10 client.25727 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68ab400640 1 -- 172.21.15.6:0/1152034001 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cb9021cb0 con 0x561cb90de540 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68ab400640 10 client.25727 handle_command_reply: tid=0 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 client.25727 shutdown 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 msgr2=0x561cb90cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f68b80cba10 tx=0x7f68b809a700 comp rx=0 tx=0).stop 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 shutdown_connections 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb90de540 0x561cb90e0960 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbad0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 --2- 172.21.15.6:0/1152034001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68ab400640 0 client.25727 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 >> 172.21.15.6:0/1152034001 conn(0x561cb908c480 msgr2=0x561cb90c1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 shutdown_connections 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 1 -- 172.21.15.6:0/1152034001 wait complete. 2024-02-26T06:28:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.830+0000 7f68d35a0740 20 client.25727 trim_cache size 0 max 16384 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.844+0000 7f68d35a0740 1 Processor -- start 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.844+0000 7f68d35a0740 1 -- start start 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68d35a0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb8d73e60 con 0x561cb90c4fc0 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68d35a0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb90c0a40 con 0x561cb90c5d30 2024-02-26T06:28:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68d35a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb90e5b70 con 0x561cb90ce540 2024-02-26T06:28:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68cdc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68cd200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68cdc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37104/0 (socket says 172.21.15.6:37104) 2024-02-26T06:28:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68cdc00640 1 -- 172.21.15.6:0/4233992297 learned_addr learned my addr 172.21.15.6:0/4233992297 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68cd200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48748/0 (socket says 172.21.15.6:48748) 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.845+0000 7f68ce600640 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68cd200640 1 -- 172.21.15.6:0/4233992297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 msgr2=0x561cb90d5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68cd200640 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90d5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68cd200640 1 -- 172.21.15.6:0/4233992297 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 msgr2=0x561cb90cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68cd200640 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68cd200640 1 -- 172.21.15.6:0/4233992297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb8fc6500 con 0x561cb90c5d30 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68cd200640 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbe30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005fd0 tx=0x7f68b80cb950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f06f1e8c5d72e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68b6000640 1 -- 172.21.15.6:0/4233992297 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b809a7f0 con 0x561cb90c5d30 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68b6000640 1 -- 172.21.15.6:0/4233992297 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68b8079ad0 con 0x561cb90c5d30 2024-02-26T06:28:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.846+0000 7f68b6000640 1 -- 172.21.15.6:0/4233992297 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80cbf20 con 0x561cb90c5d30 2024-02-26T06:28:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 -- 172.21.15.6:0/4233992297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 msgr2=0x561cb90cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbe30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005fd0 tx=0x7f68b80cb950 comp rx=0 tx=0).stop 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 -- 172.21.15.6:0/4233992297 shutdown_connections 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90ce540 0x561cb90d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90c5d30 0x561cb90cbe30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 --2- 172.21.15.6:0/4233992297 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c4fc0 0x561cb90cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 -- 172.21.15.6:0/4233992297 >> 172.21.15.6:0/4233992297 conn(0x561cb908c480 msgr2=0x561cb90bf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 -- 172.21.15.6:0/4233992297 shutdown_connections 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 -- 172.21.15.6:0/4233992297 wait complete. 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.847+0000 7f68d35a0740 1 -- start start 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 Processor -- start 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c4fc0 0x561cb90d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 0x561cb90c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90ce540 0x561cb90c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb90c8b60 con 0x561cb90c5d30 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb90c8cd0 con 0x561cb90ce540 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb9134a60 con 0x561cb90c4fc0 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68d35a0740 10 client.0 _ll_register_callbacks cb 0x7f68b8009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68cd200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 0x561cb90c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68cd200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 0x561cb90c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37120/0 (socket says 172.21.15.6:37120) 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68cd200640 1 -- 172.21.15.6:0/2507185679 learned_addr learned my addr 172.21.15.6:0/2507185679 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.848+0000 7f68cdc00640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c4fc0 0x561cb90d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68ce600640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90ce540 0x561cb90c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68cd200640 1 -- 172.21.15.6:0/2507185679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c4fc0 msgr2=0x561cb90d5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68cd200640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c4fc0 0x561cb90d5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68cd200640 1 -- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90ce540 msgr2=0x561cb90c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68cd200640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90ce540 0x561cb90c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68cd200640 1 -- 172.21.15.6:0/2507185679 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cb9029c20 con 0x561cb90c5d30 2024-02-26T06:28:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68cd200640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 0x561cb90c7f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f68b809a9b0 tx=0x7f68b800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be6579a6f6bcd9e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68ab400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68ab400640 1 -- 172.21.15.6:0/2507185679 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8096c50 con 0x561cb90c5d30 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68ab400640 1 -- 172.21.15.6:0/2507185679 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f68b80a33a0 con 0x561cb90c5d30 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.849+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cb9050d00 con 0x561cb90c5d30 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68ab400640 1 -- 172.21.15.6:0/2507185679 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68b809a7f0 con 0x561cb90c5d30 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561cb8fec290 con 0x561cb90c5d30 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68ab400640 1 -- 172.21.15.6:0/2507185679 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561cb9050d00 con 0x561cb90c5d30 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68ab400640 1 -- 172.21.15.6:0/2507185679 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f68b809fed0 con 0x561cb90c5d30 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 20 client.15594 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb9135c50 0x561cb9138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 4 client.15594 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:28:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.850+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561cb8d2b600 con 0x561cb9135c50 2024-02-26T06:28:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.851+0000 7f68cdc00640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb9135c50 0x561cb9138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.851+0000 7f68cdc00640 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb9135c50 0x561cb9138040 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60fac7a5dca9120f server_cookie=cfc567d8933733b1 in_seq=0 out_seq=0 2024-02-26T06:28:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.851+0000 7f68ab400640 10 client.15594 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.859+0000 7f68ab400640 1 -- 172.21.15.6:0/2507185679 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561cb8d2b600 con 0x561cb9135c50 2024-02-26T06:28:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.859+0000 7f68ab400640 10 client.15594 handle_command_reply: tid=0 2024-02-26T06:28:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.859+0000 7f68d35a0740 1 client.15594 shutdown 2024-02-26T06:28:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.859+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 msgr2=0x561cb90c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.859+0000 7f68d35a0740 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 0x561cb90c7f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f68b809a9b0 tx=0x7f68b800ce20 comp rx=0 tx=0).stop 2024-02-26T06:28:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 shutdown_connections 2024-02-26T06:28:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x561cb9135c50 0x561cb9138040 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cb90ce540 0x561cb90c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561cb90c5d30 0x561cb90c7f20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 --2- 172.21.15.6:0/2507185679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cb90c4fc0 0x561cb90d5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 >> 172.21.15.6:0/2507185679 conn(0x561cb908c480 msgr2=0x561cb90c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68ab400640 0 client.15594 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 shutdown_connections 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 1 -- 172.21.15.6:0/2507185679 wait complete. 2024-02-26T06:28:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.860+0000 7f68d35a0740 20 client.15594 trim_cache size 0 max 16384 2024-02-26T06:28:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:28:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:28:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:28:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f68b00a22a0 msgr2=0x7f68b00a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f68b00a22a0 0x7f68b00a4760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68c002cfd0 tx=0x7f68c002bfa0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073e50 msgr2=0x7f68c819f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073e50 0x7f68c819f960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f68b800e790 tx=0x7f68b800ec50 comp rx=0 tx=0).stop 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 shutdown_connections 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f68b00a22a0 0x7f68b00a4760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68c810d820 0x7f68c8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c8073e50 0x7f68c819f960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 --2- 172.21.15.6:0/1488672313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c8073530 0x7f68c819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 >> 172.21.15.6:0/1488672313 conn(0x7f68c80fca90 msgr2=0x7f68c80fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 shutdown_connections 2024-02-26T06:28:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:28:48.861+0000 7f68cf000640 1 -- 172.21.15.6:0/1488672313 wait complete. 2024-02-26T06:28:50.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:49 smithi006 ceph-mon[31790]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:28:50.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:49 smithi167 ceph-mon[36229]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:28:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:49 smithi090 ceph-mon[35681]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:28:52.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:51 smithi006 ceph-mon[31790]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-02-26T06:28:52.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:51 smithi167 ceph-mon[36229]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-02-26T06:28:52.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:51 smithi090 ceph-mon[35681]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-02-26T06:28:54.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:53 smithi090 ceph-mon[35681]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-02-26T06:28:54.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:53 smithi006 ceph-mon[31790]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-02-26T06:28:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:53 smithi167 ceph-mon[36229]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-02-26T06:28:55.153 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 67% (7721/11369) Updating files: 68% (7731/11369) Updating files: 69% (7845/11369) Updating files: 70% (7959/11369) Updating files: 71% (8072/11369) Updating files: 72% (8186/11369) Updating files: 73% (8300/11369) Updating files: 74% (8414/11369) Updating files: 75% (8527/11369) Updating files: 76% (8641/11369) Updating files: 77% (8755/11369) Updating files: 78% (8868/11369) Updating files: 79% (8982/11369) Updating files: 80% (9096/11369) Updating files: 81% (9209/11369) Updating files: 82% (9323/11369) Updating files: 83% (9437/11369) Updating files: 84% (9550/11369) Updating files: 85% (9664/11369) Updating files: 86% (9778/11369) Updating files: 87% (9892/11369) Updating files: 88% (10005/11369) Updating files: 89% (10119/11369) Updating files: 90% (10233/11369) Updating files: 91% (10346/11369) Updating files: 92% (10460/11369) Updating files: 93% (10574/11369) Updating files: 94% (10687/11369) Updating files: 95% (10801/11369) Updating files: 96% (10915/11369) Updating files: 97% (11028/11369) Updating files: 98% (11142/11369) Updating files: 99% (11256/11369) Updating files: 100% (11369/11369) Updating files: 100% (11369/11369), done. 2024-02-26T06:28:56.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:55 smithi006 ceph-mon[31790]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-02-26T06:28:56.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:55 smithi167 ceph-mon[36229]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'ba6fbbfee45488c6bcf0f2f12a3da33f007f0031'. 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-02-26T06:28:56.329 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:28:56.330 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at ba6fbbfee45 qa/cephfs: add `MON_DOWN` to ignorelist 2024-02-26T06:28:56.340 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-26T06:28:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:55 smithi090 ceph-mon[35681]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-02-26T06:28:56.409 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-26T06:28:56.418 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-26T06:28:56.418 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-26T06:28:57.383 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-26T06:28:57.559 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-26T06:28:57.794 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-26T06:28:57.794 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-26T06:28:57.794 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-26T06:28:57.794 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-26T06:28:57.794 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:28:57.794 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-26T06:28:57.856 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-02-26T06:28:57.858 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-02-26T06:28:57.858 DEBUG:teuthology.orchestra.run.smithi006:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-02-26T06:28:57.938 INFO:tasks.workunit.client.0.smithi006.stderr:+ set -e 2024-02-26T06:28:57.938 INFO:tasks.workunit.client.0.smithi006.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-02-26T06:28:57.945 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into 'xfstests-dev'... 2024-02-26T06:28:58.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:57 smithi167 ceph-mon[36229]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T06:28:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:57 smithi090 ceph-mon[35681]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T06:28:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:57 smithi006 ceph-mon[31790]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T06:28:59.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:28:59.677 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.885+0000 7f4c7fa00640 1 -- 172.21.15.6:0/3669851787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78101d30 msgr2=0x7f4c78105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.885+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/3669851787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78101d30 0x7f4c78105d70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009890 tx=0x7f4c6402bb70 comp rx=0 tx=0).stop 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 -- 172.21.15.6:0/3669851787 shutdown_connections 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/3669851787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c78106450 0x7f4c7810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/3669851787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78101d30 0x7f4c78105d70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/3669851787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 0x7f4c78101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 -- 172.21.15.6:0/3669851787 >> 172.21.15.6:0/3669851787 conn(0x7f4c780fcc20 msgr2=0x7f4c780ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:59.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 -- 172.21.15.6:0/3669851787 shutdown_connections 2024-02-26T06:28:59.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.886+0000 7f4c7fa00640 1 -- 172.21.15.6:0/3669851787 wait complete. 2024-02-26T06:28:59.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.887+0000 7f4c7fa00640 1 Processor -- start 2024-02-26T06:28:59.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.887+0000 7f4c7fa00640 1 -- start start 2024-02-26T06:28:59.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 0x7f4c7819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7fa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 0x7f4c7819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7fa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c78197630 0x7f4c78197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7fa00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c78198100 con 0x7f4c78197630 2024-02-26T06:28:59.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7fa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c78198270 con 0x7f4c78106450 2024-02-26T06:28:59.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c78198430 con 0x7f4c78101380 2024-02-26T06:28:59.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 0x7f4c7819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 0x7f4c7819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 0x7f4c7819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60496/0 (socket says 172.21.15.6:60496) 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c77e00640 1 -- 172.21.15.6:0/1840156373 learned_addr learned my addr 172.21.15.6:0/1840156373 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.888+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c78197630 0x7f4c78197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c7d000640 1 -- 172.21.15.6:0/1840156373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 msgr2=0x7f4c7819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c7d000640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 0x7f4c7819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c7d000640 1 -- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c78197630 msgr2=0x7f4c78197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c7d000640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c78197630 0x7f4c78197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c7d000640 1 -- 172.21.15.6:0/1840156373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64009490 con 0x7f4c78101380 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c77e00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 0x7f4c7819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:59.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.889+0000 7f4c7d000640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 0x7f4c7819d570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e790 tx=0x7f4c6c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:59.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.890+0000 7f4c75600640 1 -- 172.21.15.6:0/1840156373 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00ccc0 con 0x7f4c78101380 2024-02-26T06:28:59.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.890+0000 7f4c75600640 1 -- 172.21.15.6:0/1840156373 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c00eea0 con 0x7f4c78101380 2024-02-26T06:28:59.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.890+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c781a43d0 con 0x7f4c78101380 2024-02-26T06:28:59.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.890+0000 7f4c75600640 1 -- 172.21.15.6:0/1840156373 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c018730 con 0x7f4c78101380 2024-02-26T06:28:59.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.890+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c781a48b0 con 0x7f4c78101380 2024-02-26T06:28:59.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.895+0000 7f4c829c6740 1 Processor -- start 2024-02-26T06:28:59.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.895+0000 7f4c829c6740 1 -- start start 2024-02-26T06:28:59.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.895+0000 7f4c75600640 1 -- 172.21.15.6:0/1840156373 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f4c6c010430 con 0x7f4c78101380 2024-02-26T06:28:59.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.895+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 0x557ea3c67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.895+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c68710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.895+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c71600 0x557ea3c739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c829c6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3bd7120 con 0x557ea3c71600 2024-02-26T06:28:59.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c829c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3bd4650 con 0x557ea3c68290 2024-02-26T06:28:59.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c829c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3bcfe30 con 0x557ea3c68f70 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 0x557ea3c67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c71600 0x557ea3c739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 0x557ea3c67cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39366/0 (socket says 172.21.15.6:39366) 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c7da00640 1 -- 172.21.15.6:0/1554917580 learned_addr learned my addr 172.21.15.6:0/1554917580 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.896+0000 7f4c7d000640 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c68710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7da00640 1 -- 172.21.15.6:0/1554917580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 msgr2=0x557ea3c68710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c68710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7da00640 1 -- 172.21.15.6:0/1554917580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c71600 msgr2=0x557ea3c739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c71600 0x557ea3c739f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7da00640 1 -- 172.21.15.6:0/1554917580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea3b89030 con 0x557ea3c68f70 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c75600640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4c600a22c0 0x7f4c600a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 0x557ea3c67cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009f50 tx=0x7f4c700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0561f06aac8a128 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c75600640 1 -- 172.21.15.6:0/1840156373 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4c6c014070 con 0x7f4c78101380 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.897+0000 7f4c7d000640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4c600a22c0 0x7f4c600a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.898+0000 7f4c6a000640 1 -- 172.21.15.6:0/1554917580 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70004b20 con 0x557ea3c68f70 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.898+0000 7f4c6a000640 1 -- 172.21.15.6:0/1554917580 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c70004c80 con 0x557ea3c68f70 2024-02-26T06:28:59.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.898+0000 7f4c6a000640 1 -- 172.21.15.6:0/1554917580 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70005020 con 0x557ea3c68f70 2024-02-26T06:28:59.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 -- 172.21.15.6:0/1554917580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 msgr2=0x557ea3c67cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 0x557ea3c67cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009f50 tx=0x7f4c700118d0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c7d000640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4c600a22c0 0x7f4c600a4780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0cbac0 tx=0x7f4c6c0d1c50 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 -- 172.21.15.6:0/1554917580 shutdown_connections 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c71600 0x557ea3c739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c68710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1554917580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c68f70 0x557ea3c67cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 -- 172.21.15.6:0/1554917580 >> 172.21.15.6:0/1554917580 conn(0x557ea3c2f570 msgr2=0x557ea3c62ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 -- 172.21.15.6:0/1554917580 shutdown_connections 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 -- 172.21.15.6:0/1554917580 wait complete. 2024-02-26T06:28:59.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.899+0000 7f4c829c6740 1 -- start start 2024-02-26T06:28:59.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 Processor -- start 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 0x557ea3c7b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68f70 0x557ea3c7ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c75950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3b74e90 con 0x557ea3c68290 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3b74b30 con 0x557ea3c68f70 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea393e1c0 con 0x557ea3c71600 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.900+0000 7f4c829c6740 10 client.0 _ll_register_callbacks cb 0x557ea3b88930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 0x557ea3c7b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 0x557ea3c7b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45584/0 (socket says 172.21.15.6:45584) 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 -- 172.21.15.6:0/1623079712 learned_addr learned my addr 172.21.15.6:0/1623079712 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c77e00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68f70 0x557ea3c7ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c75950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 -- 172.21.15.6:0/1623079712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 msgr2=0x557ea3c75950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c75950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 -- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68f70 msgr2=0x557ea3c7ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68f70 0x557ea3c7ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c7da00640 1 -- 172.21.15.6:0/1623079712 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c70009c00 con 0x557ea3c68290 2024-02-26T06:28:59.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.901+0000 7f4c77e00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68f70 0x557ea3c7ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.902+0000 7f4c7da00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 0x557ea3c7b540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c700049d0 tx=0x7f4c700044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801a2f8b6de82e1f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.902+0000 7f4c5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.902+0000 7f4c5ea00640 1 -- 172.21.15.6:0/1623079712 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70027530 con 0x557ea3c68290 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.902+0000 7f4c5ea00640 1 -- 172.21.15.6:0/1623079712 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4c70025460 con 0x557ea3c68290 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.902+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ea38cf7e0 con 0x557ea3c68290 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c5ea00640 1 -- 172.21.15.6:0/1623079712 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c70027690 con 0x557ea3c68290 2024-02-26T06:28:59.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 10 client.15606 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557ea3bce4b0 con 0x557ea3c68290 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c5ea00640 1 -- 172.21.15.6:0/1623079712 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c70033080 con 0x557ea3c68290 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c5ea00640 1 -- 172.21.15.6:0/1623079712 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4c7002e070 con 0x557ea3c68290 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 20 client.15606 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3c85a00 0x557ea3c87df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 4 client.15606 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.903+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ea390c720 con 0x557ea3c85a00 2024-02-26T06:28:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.904+0000 7f4c77e00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3c85a00 0x557ea3c87df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.904+0000 7f4c77e00640 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3c85a00 0x557ea3c87df0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f7dde71e34a303 server_cookie=6c728f71788642cb in_seq=0 out_seq=0 2024-02-26T06:28:59.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.904+0000 7f4c5ea00640 10 client.15606 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:59.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.905+0000 7f4c5ea00640 1 -- 172.21.15.6:0/1623079712 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ea390c720 con 0x557ea3c85a00 2024-02-26T06:28:59.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.905+0000 7f4c5ea00640 10 client.15606 handle_command_reply: tid=0 2024-02-26T06:28:59.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.905+0000 7f4c829c6740 1 client.15606 shutdown 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.905+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 msgr2=0x557ea3c7b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.905+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 0x557ea3c7b540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c700049d0 tx=0x7f4c700044a0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 shutdown_connections 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3c85a00 0x557ea3c87df0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c75950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68f70 0x557ea3c7ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 --2- 172.21.15.6:0/1623079712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68290 0x557ea3c7b540 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 >> 172.21.15.6:0/1623079712 conn(0x557ea3c2f570 msgr2=0x557ea3c64a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c5ea00640 0 client.15606 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 shutdown_connections 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 1 -- 172.21.15.6:0/1623079712 wait complete. 2024-02-26T06:28:59.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.906+0000 7f4c829c6740 20 client.15606 trim_cache size 0 max 16384 2024-02-26T06:28:59.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.923+0000 7f4c829c6740 1 Processor -- start 2024-02-26T06:28:59.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.923+0000 7f4c829c6740 1 -- start start 2024-02-26T06:28:59.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c75720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c6fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c829c6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3c68bd0 con 0x557ea3c68f70 2024-02-26T06:28:59.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c829c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3c638b0 con 0x557ea3c68290 2024-02-26T06:28:59.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c829c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3c672c0 con 0x557ea3c71600 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c77e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c75720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c751e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60518/0 (socket says 172.21.15.6:60518) 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c7da00640 1 -- 172.21.15.6:0/3150714504 learned_addr learned my addr 172.21.15.6:0/3150714504 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.924+0000 7f4c7da00640 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c6fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c7da00640 1 -- 172.21.15.6:0/3150714504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 msgr2=0x557ea3c6fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c7da00640 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c6fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c7da00640 1 -- 172.21.15.6:0/3150714504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 msgr2=0x557ea3c75720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c7da00640 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c75720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c7da00640 1 -- 172.21.15.6:0/3150714504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea3b6a920 con 0x557ea3c68290 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c77e00640 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c75720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c7da00640 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c751e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005f40 tx=0x7f4c70000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a87c03422c1d15 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c69600640 1 -- 172.21.15.6:0/3150714504 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70004000 con 0x557ea3c68290 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c69600640 1 -- 172.21.15.6:0/3150714504 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557ea38cf7e0 con 0x557ea3c68290 2024-02-26T06:28:59.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.925+0000 7f4c69600640 1 -- 172.21.15.6:0/3150714504 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000f910 con 0x557ea3c68290 2024-02-26T06:28:59.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 -- 172.21.15.6:0/3150714504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 msgr2=0x557ea3c751e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c751e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005f40 tx=0x7f4c70000f40 comp rx=0 tx=0).stop 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 -- 172.21.15.6:0/3150714504 shutdown_connections 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c6fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c75720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 --2- 172.21.15.6:0/3150714504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c751e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.926+0000 7f4c829c6740 1 -- 172.21.15.6:0/3150714504 >> 172.21.15.6:0/3150714504 conn(0x557ea3c2f570 msgr2=0x557ea3c62810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.927+0000 7f4c829c6740 1 -- 172.21.15.6:0/3150714504 shutdown_connections 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.927+0000 7f4c829c6740 1 -- 172.21.15.6:0/3150714504 wait complete. 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.927+0000 7f4c829c6740 1 -- start start 2024-02-26T06:28:59.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.927+0000 7f4c829c6740 1 Processor -- start 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c7b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c7b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3c77ce0 con 0x557ea3c68f70 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3c77e50 con 0x557ea3c68290 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea3c78010 con 0x557ea3c71600 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c829c6740 10 client.0 _ll_register_callbacks cb 0x7f4c700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c7b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c7da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c7b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60522/0 (socket says 172.21.15.6:60522) 2024-02-26T06:28:59.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c7da00640 1 -- 172.21.15.6:0/919710685 learned_addr learned my addr 172.21.15.6:0/919710685 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.928+0000 7f4c7da00640 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c7da00640 1 -- 172.21.15.6:0/919710685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 msgr2=0x557ea3c776a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c7da00640 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c776a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c7da00640 1 -- 172.21.15.6:0/919710685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 msgr2=0x557ea3c7b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c7da00640 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c7b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c7da00640 1 -- 172.21.15.6:0/919710685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7810e830 con 0x557ea3c68290 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c7da00640 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c7b200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c70011ed0 tx=0x7f4c70025a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc246aa0451cb51 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:28:59.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.929+0000 7f4c5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:28:59.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.932+0000 7f4c5ea00640 1 -- 172.21.15.6:0/919710685 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c700042c0 con 0x557ea3c68290 2024-02-26T06:28:59.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.932+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c70004c80 con 0x557ea3c68290 2024-02-26T06:28:59.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.932+0000 7f4c5ea00640 1 -- 172.21.15.6:0/919710685 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4c70039070 con 0x557ea3c68290 2024-02-26T06:28:59.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.933+0000 7f4c5ea00640 1 -- 172.21.15.6:0/919710685 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c70004000 con 0x557ea3c68290 2024-02-26T06:28:59.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.933+0000 7f4c829c6740 10 client.25751 fetch_fsmap learned FSMap version 27 2024-02-26T06:28:59.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.933+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557ea3bce4b0 con 0x557ea3c68290 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.933+0000 7f4c5ea00640 1 -- 172.21.15.6:0/919710685 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c70004c80 con 0x557ea3c68290 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c5ea00640 1 -- 172.21.15.6:0/919710685 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4c7003c070 con 0x557ea3c68290 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c829c6740 10 client.25751 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c829c6740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c829c6740 20 client.25751 populate_metadata read hostname 'smithi006' 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c829c6740 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3ce1290 0x557ea3ce3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c829c6740 4 client.25751 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:28:59.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ea395b890 con 0x557ea3ce1290 2024-02-26T06:28:59.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.934+0000 7f4c77e00640 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3ce1290 0x557ea3ce3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:28:59.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c77e00640 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3ce1290 0x557ea3ce3680 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24d7b367075e976 server_cookie=35b132613204014 in_seq=0 out_seq=0 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c5ea00640 10 client.25751 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c5ea00640 1 -- 172.21.15.6:0/919710685 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557ea395b890 con 0x557ea3ce1290 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c5ea00640 10 client.25751 handle_command_reply: tid=0 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 client.25751 shutdown 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 msgr2=0x557ea3c7b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c7b200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c70011ed0 tx=0x7f4c70025a80 comp rx=0 tx=0).stop 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 shutdown_connections 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x557ea3ce1290 0x557ea3ce3680 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ea3c71600 0x557ea3c776a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557ea3c68f70 0x557ea3c7b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 --2- 172.21.15.6:0/919710685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557ea3c68290 0x557ea3c7b200 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 >> 172.21.15.6:0/919710685 conn(0x557ea3c2f570 msgr2=0x557ea3c75160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:59.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.935+0000 7f4c5ea00640 0 client.25751 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:28:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.936+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 shutdown_connections 2024-02-26T06:28:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.936+0000 7f4c829c6740 1 -- 172.21.15.6:0/919710685 wait complete. 2024-02-26T06:28:59.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.936+0000 7f4c829c6740 20 client.25751 trim_cache size 0 max 16384 2024-02-26T06:28:59.937 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:28:59.937 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-02-26T06:28:59.938 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-02-26T06:28:59.938 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:28:59.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4c600a22c0 msgr2=0x7f4c600a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4c600a22c0 0x7f4c600a4780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0cbac0 tx=0x7f4c6c0d1c50 comp rx=0 tx=0).stop 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 msgr2=0x7f4c7819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 0x7f4c7819d570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e790 tx=0x7f4c6c00ec50 comp rx=0 tx=0).stop 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 shutdown_connections 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4c600a22c0 0x7f4c600a4780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c78197630 0x7f4c78197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c78106450 0x7f4c7819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 --2- 172.21.15.6:0/1840156373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c78101380 0x7f4c7819d570 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 >> 172.21.15.6:0/1840156373 conn(0x7f4c780fcc20 msgr2=0x7f4c780fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 shutdown_connections 2024-02-26T06:28:59.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:28:59.937+0000 7f4c7fa00640 1 -- 172.21.15.6:0/1840156373 wait complete. 2024-02-26T06:28:59.962 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:28:59.962 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:29:00.038 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:28:59 smithi006 ceph-mon[31790]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-02-26T06:29:00.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 -- 172.21.15.6:0/1434016321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073530 msgr2=0x7f6038073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 --2- 172.21.15.6:0/1434016321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073530 0x7f6038073910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6028009860 tx=0x7f602802bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 -- 172.21.15.6:0/1434016321 shutdown_connections 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 --2- 172.21.15.6:0/1434016321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603810d8d0 0x7f603810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 --2- 172.21.15.6:0/1434016321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073e50 0x7f603810d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 --2- 172.21.15.6:0/1434016321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073530 0x7f6038073910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 -- 172.21.15.6:0/1434016321 >> 172.21.15.6:0/1434016321 conn(0x7f60380fcb40 msgr2=0x7f60380fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 -- 172.21.15.6:0/1434016321 shutdown_connections 2024-02-26T06:29:00.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.161+0000 7f603ca00640 1 -- 172.21.15.6:0/1434016321 wait complete. 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.162+0000 7f603ca00640 1 Processor -- start 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.162+0000 7f603ca00640 1 -- start start 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f603ca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 0x7f603819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f603ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073e50 0x7f603819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f603ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603810d8d0 0x7f60381996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f603ca00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6038199e50 con 0x7f6038073530 2024-02-26T06:29:00.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f603ca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6038199fc0 con 0x7f603810d8d0 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f603ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819a180 con 0x7f6038073e50 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 0x7f603819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 0x7f603819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45622/0 (socket says 172.21.15.6:45622) 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037400640 1 -- 172.21.15.6:0/1516312372 learned_addr learned my addr 172.21.15.6:0/1516312372 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6036a00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073e50 0x7f603819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037e00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603810d8d0 0x7f60381996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037400640 1 -- 172.21.15.6:0/1516312372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073e50 msgr2=0x7f603819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037400640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073e50 0x7f603819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.163+0000 7f6037400640 1 -- 172.21.15.6:0/1516312372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603810d8d0 msgr2=0x7f60381996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f6037400640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603810d8d0 0x7f60381996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f6037400640 1 -- 172.21.15.6:0/1516312372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028009490 con 0x7f6038073530 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f6037400640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 0x7f603819f5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6028009980 tx=0x7f60280346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602802c7d0 con 0x7f6038073530 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60280044b0 con 0x7f6038073530 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602803f440 con 0x7f6038073530 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.164+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603819a400 con 0x7f6038073530 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.165+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60381a6530 con 0x7f6038073530 2024-02-26T06:29:00.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.166+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6038074be0 con 0x7f6038073530 2024-02-26T06:29:00.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.168+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f602803b040 con 0x7f6038073530 2024-02-26T06:29:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.170+0000 7f602fe00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f601c0a2400 0x7f601c0a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.171+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f60280effd0 con 0x7f6038073530 2024-02-26T06:29:00.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.171+0000 7f6036a00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f601c0a2400 0x7f601c0a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.172+0000 7f6036a00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f601c0a2400 0x7f601c0a48c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f603819aec0 tx=0x7f6024009260 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.176+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60280b8b80 con 0x7f6038073530 2024-02-26T06:29:00.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:28:59 smithi167 ceph-mon[36229]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-02-26T06:29:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:28:59 smithi090 ceph-mon[35681]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-02-26T06:29:00.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.528+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6038108a50 con 0x7f6038073530 2024-02-26T06:29:00.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.529+0000 7f602fe00640 1 -- 172.21.15.6:0/1516312372 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f60280bc240 con 0x7f6038073530 2024-02-26T06:29:00.529 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:29:00.530 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:27:45.824981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:29:00.530 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 27 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f601c0a2400 msgr2=0x7f601c0a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f601c0a2400 0x7f601c0a48c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f603819aec0 tx=0x7f6024009260 comp rx=0 tx=0).stop 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 msgr2=0x7f603819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 0x7f603819f5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6028009980 tx=0x7f60280346f0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 shutdown_connections 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f601c0a2400 0x7f601c0a48c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603810d8d0 0x7f60381996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6038073e50 0x7f603819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 --2- 172.21.15.6:0/1516312372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6038073530 0x7f603819f5b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 >> 172.21.15.6:0/1516312372 conn(0x7f60380fcb40 msgr2=0x7f60380fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 shutdown_connections 2024-02-26T06:29:00.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.532+0000 7f603ca00640 1 -- 172.21.15.6:0/1516312372 wait complete. 2024-02-26T06:29:00.551 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:00.551 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-26T06:29:00.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 -- 172.21.15.6:0/33429921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec10d960 msgr2=0x7f1fec10fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 --2- 172.21.15.6:0/33429921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec10d960 0x7f1fec10fdb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009ca0 tx=0x7f1fe802c190 comp rx=0 tx=0).stop 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 -- 172.21.15.6:0/33429921 shutdown_connections 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 --2- 172.21.15.6:0/33429921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec10d960 0x7f1fec10fdb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 --2- 172.21.15.6:0/33429921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fec073ed0 0x7f1fec10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 --2- 172.21.15.6:0/33429921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec0735b0 0x7f1fec073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 -- 172.21.15.6:0/33429921 >> 172.21.15.6:0/33429921 conn(0x7f1fec0fcc10 msgr2=0x7f1fec0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 -- 172.21.15.6:0/33429921 shutdown_connections 2024-02-26T06:29:00.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.785+0000 7f1ff4800640 1 -- 172.21.15.6:0/33429921 wait complete. 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.786+0000 7f1ff4800640 1 Processor -- start 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.786+0000 7f1ff4800640 1 -- start start 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff4800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 0x7f1fec19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff4800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fec073ed0 0x7f1fec19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff4800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec10d960 0x7f1fec199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff4800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec199e20 con 0x7f1fec073ed0 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff4800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec199f90 con 0x7f1fec0735b0 2024-02-26T06:29:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff4800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec19a150 con 0x7f1fec10d960 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 0x7f1fec19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 0x7f1fec19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60566/0 (socket says 172.21.15.6:60566) 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 -- 172.21.15.6:0/2857022317 learned_addr learned my addr 172.21.15.6:0/2857022317 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1400640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fec073ed0 0x7f1fec19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff2800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec10d960 0x7f1fec199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 -- 172.21.15.6:0/2857022317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec10d960 msgr2=0x7f1fec199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec10d960 0x7f1fec199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 -- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fec073ed0 msgr2=0x7f1fec19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fec073ed0 0x7f1fec19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff1e00640 1 -- 172.21.15.6:0/2857022317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe0009490 con 0x7f1fec0735b0 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.787+0000 7f1ff2800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec10d960 0x7f1fec199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.788+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 0x7f1fec19f610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000ea60 tx=0x7f1fe000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.788+0000 7f1fe6a00640 1 -- 172.21.15.6:0/2857022317 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000cde0 con 0x7f1fec0735b0 2024-02-26T06:29:00.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.788+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc009490 con 0x7f1fec0735b0 2024-02-26T06:29:00.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.788+0000 7f1fe6a00640 1 -- 172.21.15.6:0/2857022317 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fe00035f0 con 0x7f1fec0735b0 2024-02-26T06:29:00.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.788+0000 7f1fe6a00640 1 -- 172.21.15.6:0/2857022317 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0010690 con 0x7f1fec0735b0 2024-02-26T06:29:00.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.788+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe8009900 con 0x7f1fec0735b0 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.791+0000 7f1ff781f740 1 Processor -- start 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.791+0000 7f1ff781f740 1 -- start start 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1fe6a00640 1 -- 172.21.15.6:0/2857022317 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f1fe0003af0 con 0x7f1fec0735b0 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff10b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff781f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff101e1e0 con 0x55eff10b9110 2024-02-26T06:29:00.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff781f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff101b710 con 0x55eff10b0530 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff781f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff1016ef0 con 0x55eff10afc10 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff10b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10bb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45650/0 (socket says 172.21.15.6:45650) 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 -- 172.21.15.6:0/3730613088 learned_addr learned my addr 172.21.15.6:0/3730613088 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff10b8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60578/0 (socket says 172.21.15.6:60578) 2024-02-26T06:29:00.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff2800640 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 -- 172.21.15.6:0/3730613088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 msgr2=0x55eff10afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 -- 172.21.15.6:0/3730613088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 msgr2=0x55eff10b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff10b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1400640 1 -- 172.21.15.6:0/3730613088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eff0fd8dc0 con 0x55eff10b9110 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.792+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff10b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1ff1400640 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10bb500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009db0 tx=0x7f1fdc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70325a1232da02a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1fe6a00640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f1fd40a22c0 0x7f1fd40a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1fdb000640 1 -- 172.21.15.6:0/3730613088 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc019410 con 0x55eff10b9110 2024-02-26T06:29:00.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1fe6a00640 1 -- 172.21.15.6:0/2857022317 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f1fe0014070 con 0x7f1fec0735b0 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1fdb000640 1 -- 172.21.15.6:0/3730613088 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc019a50 con 0x55eff10b9110 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1fdb000640 1 -- 172.21.15.6:0/3730613088 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc018a80 con 0x55eff10b9110 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.793+0000 7f1ff2800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f1fd40a22c0 0x7f1fd40a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 -- 172.21.15.6:0/3730613088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 msgr2=0x55eff10bb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10bb500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009db0 tx=0x7f1fdc011810 comp rx=0 tx=0).stop 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff2800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f1fd40a22c0 0x7f1fd40a4780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8006010 tx=0x7f1fe8004a40 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 -- 172.21.15.6:0/3730613088 shutdown_connections 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10bb500 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff10b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 --2- 172.21.15.6:0/3730613088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 -- 172.21.15.6:0/3730613088 >> 172.21.15.6:0/3730613088 conn(0x55eff10aa340 msgr2=0x55eff10aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 -- 172.21.15.6:0/3730613088 shutdown_connections 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.794+0000 7f1ff781f740 1 -- 172.21.15.6:0/3730613088 wait complete. 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 -- start start 2024-02-26T06:29:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 Processor -- start 2024-02-26T06:29:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b9110 0x55eff10b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff0fbc120 con 0x55eff10b0530 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff0fbbdc0 con 0x55eff10b9110 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.795+0000 7f1ff781f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff0d851c0 con 0x55eff10afc10 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff781f740 10 client.0 _ll_register_callbacks cb 0x55eff100b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b9110 0x55eff10b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39448/0 (socket says 172.21.15.6:39448) 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45658/0 (socket says 172.21.15.6:45658) 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1e00640 1 -- 172.21.15.6:0/2692005381 learned_addr learned my addr 172.21.15.6:0/2692005381 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 -- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 msgr2=0x55eff10b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 -- 172.21.15.6:0/2692005381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b9110 msgr2=0x55eff10b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b9110 0x55eff10b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 -- 172.21.15.6:0/2692005381 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc0099e0 con 0x55eff10b0530 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff2800640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b9110 0x55eff10b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.796+0000 7f1ff1400640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b6650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc011d10 tx=0x7f1fdc0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfc1e39ae58cb2d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1fcf400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1fcf400640 1 -- 172.21.15.6:0/2692005381 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc028070 con 0x55eff10b0530 2024-02-26T06:29:00.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1fcf400640 1 -- 172.21.15.6:0/2692005381 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f1fdc0044c0 con 0x55eff10b0530 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eff0d167e0 con 0x55eff10b0530 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1fcf400640 1 -- 172.21.15.6:0/2692005381 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fdc0183b0 con 0x55eff10b0530 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1ff781f740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eff1015570 con 0x55eff10b0530 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.797+0000 7f1fcf400640 1 -- 172.21.15.6:0/2692005381 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55eff0d167e0 con 0x55eff10b0530 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1fcf400640 1 -- 172.21.15.6:0/2692005381 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fdc02b020 con 0x55eff10b0530 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff781f740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff781f740 10 client.15627 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-02-26T06:29:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff781f740 20 client.15627 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff781f740 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff10c9090 0x55eff10cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff781f740 4 client.15627 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eff0d53720 con 0x55eff10c9090 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.798+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff10c9090 0x55eff10cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.799+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff10c9090 0x55eff10cb480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e7833949125f20b server_cookie=692fb81552ffa935 in_seq=0 out_seq=0 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.799+0000 7f1fcf400640 10 client.15627 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.799+0000 7f1fcf400640 1 -- 172.21.15.6:0/2692005381 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eff0d53720 con 0x55eff10c9090 2024-02-26T06:29:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.799+0000 7f1fcf400640 10 client.15627 handle_command_reply: tid=0 2024-02-26T06:29:00.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.799+0000 7f1ff781f740 1 client.15627 shutdown 2024-02-26T06:29:00.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 msgr2=0x55eff10b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b6650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc011d10 tx=0x7f1fdc0042d0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 shutdown_connections 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff10c9090 0x55eff10cb480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b9110 0x55eff10b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b6650 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1fcf400640 0 client.15627 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 --2- 172.21.15.6:0/2692005381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 >> 172.21.15.6:0/2692005381 conn(0x55eff10aa340 msgr2=0x55eff10ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 shutdown_connections 2024-02-26T06:29:00.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 1 -- 172.21.15.6:0/2692005381 wait complete. 2024-02-26T06:29:00.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.800+0000 7f1ff781f740 20 client.15627 trim_cache size 0 max 16384 2024-02-26T06:29:00.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.815+0000 7f1ff781f740 1 Processor -- start 2024-02-26T06:29:00.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.815+0000 7f1ff781f740 1 -- start start 2024-02-26T06:29:00.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 0x55eff10b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10b9110 0x55eff10b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff781f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff10a5ab0 con 0x55eff10b0530 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff781f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff10bb500 con 0x55eff10afc10 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff781f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff10a5c10 con 0x55eff10b9110 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 0x55eff10b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10b9110 0x55eff10b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 0x55eff10b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60596/0 (socket says 172.21.15.6:60596) 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 -- 172.21.15.6:0/1610736292 learned_addr learned my addr 172.21.15.6:0/1610736292 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 -- 172.21.15.6:0/1610736292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10b9110 msgr2=0x55eff10b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10b9110 0x55eff10b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 -- 172.21.15.6:0/1610736292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 msgr2=0x55eff10b69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.816+0000 7f1ff1e00640 1 -- 172.21.15.6:0/1610736292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eff0fb1a80 con 0x55eff10afc10 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.817+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 0x55eff10b8840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00979c0 tx=0x7f1fe00d4060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab9b0c63f251722 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.817+0000 7f1fda600640 1 -- 172.21.15.6:0/1610736292 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe009a950 con 0x55eff10afc10 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.817+0000 7f1fda600640 1 -- 172.21.15.6:0/1610736292 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fe009b6f0 con 0x55eff10afc10 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.817+0000 7f1fda600640 1 -- 172.21.15.6:0/1610736292 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0099410 con 0x55eff10afc10 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 -- 172.21.15.6:0/1610736292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 msgr2=0x55eff10b8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 0x55eff10b8840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00979c0 tx=0x7f1fe00d4060 comp rx=0 tx=0).stop 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 -- 172.21.15.6:0/1610736292 shutdown_connections 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10b9110 0x55eff10b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b0530 0x55eff10b69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 --2- 172.21.15.6:0/1610736292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10afc10 0x55eff10b8840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 -- 172.21.15.6:0/1610736292 >> 172.21.15.6:0/1610736292 conn(0x55eff10aa340 msgr2=0x55eff10aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 -- 172.21.15.6:0/1610736292 shutdown_connections 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 -- 172.21.15.6:0/1610736292 wait complete. 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 -- start start 2024-02-26T06:29:00.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.818+0000 7f1ff781f740 1 Processor -- start 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff111d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10be850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff10bef50 con 0x55eff10b9110 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff10bf0c0 con 0x55eff10b0530 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff10bf280 con 0x55eff10afc10 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff781f740 10 client.0 _ll_register_callbacks cb 0x7f1fdc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff2800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10be850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff2800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10be850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45680/0 (socket says 172.21.15.6:45680) 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff2800640 1 -- 172.21.15.6:0/808919473 learned_addr learned my addr 172.21.15.6:0/808919473 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff111d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.819+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff2800640 1 -- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 msgr2=0x55eff10c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff2800640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff2800640 1 -- 172.21.15.6:0/808919473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 msgr2=0x55eff111d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff2800640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff111d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff2800640 1 -- 172.21.15.6:0/808919473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc009490 con 0x55eff10b9110 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff1400640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff111d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff2800640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10be850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009ca0 tx=0x7f1fe80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d16b3bf081745 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1fcf400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1fcf400640 1 -- 172.21.15.6:0/808919473 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8051070 con 0x55eff10b9110 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fdc019a50 con 0x55eff10b9110 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.820+0000 7f1fcf400640 1 -- 172.21.15.6:0/808919473 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f1fe8032a80 con 0x55eff10b9110 2024-02-26T06:29:00.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1fcf400640 1 -- 172.21.15.6:0/808919473 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe8032c40 con 0x55eff10b9110 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eff0fd7620 con 0x55eff10b9110 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1fcf400640 1 -- 172.21.15.6:0/808919473 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fe8062020 con 0x55eff10b9110 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1fcf400640 1 -- 172.21.15.6:0/808919473 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fe8047040 con 0x55eff10b9110 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 10 client.15633 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 20 client.15633 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff1131ea0 0x55eff11342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 4 client.15633 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eff0da2890 con 0x55eff1131ea0 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.821+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff1131ea0 0x55eff11342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:00.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1ff1e00640 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff1131ea0 0x55eff11342c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3530217d810da0a2 server_cookie=f95dfd45adc9da29 in_seq=0 out_seq=0 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1fcf400640 10 client.15633 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1fcf400640 1 -- 172.21.15.6:0/808919473 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eff0da2890 con 0x55eff1131ea0 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1fcf400640 10 client.15633 handle_command_reply: tid=0 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1ff781f740 1 client.15633 shutdown 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 msgr2=0x55eff10be850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.822+0000 7f1ff781f740 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10be850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009ca0 tx=0x7f1fe80095c0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 shutdown_connections 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55eff1131ea0 0x55eff11342c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eff10b9110 0x55eff10be850 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eff10b0530 0x55eff111d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 --2- 172.21.15.6:0/808919473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eff10afc10 0x55eff10c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1fcf400640 0 client.15633 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 >> 172.21.15.6:0/808919473 conn(0x55eff10aa340 msgr2=0x55eff10b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 shutdown_connections 2024-02-26T06:29:00.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 1 -- 172.21.15.6:0/808919473 wait complete. 2024-02-26T06:29:00.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.823+0000 7f1ff781f740 20 client.15633 trim_cache size 0 max 16384 2024-02-26T06:29:00.831 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f1fd40a22c0 msgr2=0x7f1fd40a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f1fd40a22c0 0x7f1fd40a4780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8006010 tx=0x7f1fe8004a40 comp rx=0 tx=0).stop 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 msgr2=0x7f1fec19f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 0x7f1fec19f610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000ea60 tx=0x7f1fe000ef20 comp rx=0 tx=0).stop 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 shutdown_connections 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f1fd40a22c0 0x7f1fd40a4780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fec10d960 0x7f1fec199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fec073ed0 0x7f1fec19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 --2- 172.21.15.6:0/2857022317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fec0735b0 0x7f1fec19f610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:00.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 >> 172.21.15.6:0/2857022317 conn(0x7f1fec0fcc10 msgr2=0x7f1fec0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:00.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 shutdown_connections 2024-02-26T06:29:00.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:00.824+0000 7f1ff4800640 1 -- 172.21.15.6:0/2857022317 wait complete. 2024-02-26T06:29:00.841 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:00.841 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-26T06:29:01.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 -- 172.21.15.6:0/1742046566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073530 msgr2=0x7f3a2c073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 --2- 172.21.15.6:0/1742046566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073530 0x7f3a2c073910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02bcd0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 -- 172.21.15.6:0/1742046566 shutdown_connections 2024-02-26T06:29:01.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 --2- 172.21.15.6:0/1742046566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a2c10d820 0x7f3a2c10fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 --2- 172.21.15.6:0/1742046566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c073e50 0x7f3a2c10d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 --2- 172.21.15.6:0/1742046566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073530 0x7f3a2c073910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 -- 172.21.15.6:0/1742046566 >> 172.21.15.6:0/1742046566 conn(0x7f3a2c0fca90 msgr2=0x7f3a2c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 -- 172.21.15.6:0/1742046566 shutdown_connections 2024-02-26T06:29:01.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.058+0000 7f3a2be00640 1 -- 172.21.15.6:0/1742046566 wait complete. 2024-02-26T06:29:01.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.059+0000 7f3a2be00640 1 Processor -- start 2024-02-26T06:29:01.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.059+0000 7f3a2be00640 1 -- start start 2024-02-26T06:29:01.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c073530 0x7f3a2c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 0x7f3a2c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a2c10d820 0x7f3a2c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2be00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c199e90 con 0x7f3a2c10d820 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c19a000 con 0x7f3a2c073e50 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c19a1c0 con 0x7f3a2c073530 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 0x7f3a2c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 0x7f3a2c19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60648/0 (socket says 172.21.15.6:60648) 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 -- 172.21.15.6:0/3611649728 learned_addr learned my addr 172.21.15.6:0/3611649728 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a2c10d820 0x7f3a2c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c073530 0x7f3a2c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 -- 172.21.15.6:0/3611649728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c073530 msgr2=0x7f3a2c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c073530 0x7f3a2c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 -- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a2c10d820 msgr2=0x7f3a2c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a2c10d820 0x7f3a2c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.060+0000 7f3a2a000640 1 -- 172.21.15.6:0/3611649728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c009560 con 0x7f3a2c073e50 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.061+0000 7f3a2a000640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 0x7f3a2c19fb30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800d950 tx=0x7f3a1800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.061+0000 7f3a23400640 1 -- 172.21.15.6:0/3611649728 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18009c30 con 0x7f3a2c073e50 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.061+0000 7f3a23400640 1 -- 172.21.15.6:0/3611649728 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a18009d90 con 0x7f3a2c073e50 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.061+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c19a4a0 con 0x7f3a2c073e50 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.061+0000 7f3a23400640 1 -- 172.21.15.6:0/3611649728 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800f7e0 con 0x7f3a2c073e50 2024-02-26T06:29:01.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.061+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c1a6570 con 0x7f3a2c073e50 2024-02-26T06:29:01.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.065+0000 7f3a345e6740 1 Processor -- start 2024-02-26T06:29:01.066 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.065+0000 7f3a345e6740 1 -- start start 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 0x55c4007451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a345e6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4006b31e0 con 0x55c400744dc0 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a345e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4006b0710 con 0x55c4007456e0 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a345e6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4006abef0 con 0x55c40074e2c0 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 0x55c4007451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.068 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 0x55c4007451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45732/0 (socket says 172.21.15.6:45732) 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a23400640 1 -- 172.21.15.6:0/3611649728 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f3a18010ad0 con 0x7f3a2c073e50 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60660/0 (socket says 172.21.15.6:60660) 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.066+0000 7f3a2b400640 1 -- 172.21.15.6:0/1289927813 learned_addr learned my addr 172.21.15.6:0/1289927813 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2a000640 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2b400640 1 -- 172.21.15.6:0/1289927813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 msgr2=0x55c4007506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2b400640 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2b400640 1 -- 172.21.15.6:0/1289927813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 msgr2=0x55c40074dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2b400640 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2b400640 1 -- 172.21.15.6:0/1289927813 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c40066ddc0 con 0x55c400744dc0 2024-02-26T06:29:01.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:01.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a2b400640 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 0x55c4007451a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009f70 tx=0x7f3a24010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3907e4fc86762449 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a17400640 1 -- 172.21.15.6:0/1289927813 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004d20 con 0x55c400744dc0 2024-02-26T06:29:01.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.067+0000 7f3a17400640 1 -- 172.21.15.6:0/1289927813 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a24004e80 con 0x55c400744dc0 2024-02-26T06:29:01.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a17400640 1 -- 172.21.15.6:0/1289927813 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240129b0 con 0x55c400744dc0 2024-02-26T06:29:01.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a23400640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f3a100a22c0 0x7f3a100a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a23400640 1 -- 172.21.15.6:0/3611649728 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f3a180cde70 con 0x7f3a2c073e50 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 -- 172.21.15.6:0/1289927813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 msgr2=0x55c4007451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 0x55c4007451a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009f70 tx=0x7f3a24010800 comp rx=0 tx=0).stop 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f3a100a22c0 0x7f3a100a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 -- 172.21.15.6:0/1289927813 shutdown_connections 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1289927813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c400744dc0 0x55c4007451a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.068+0000 7f3a345e6740 1 -- 172.21.15.6:0/1289927813 >> 172.21.15.6:0/1289927813 conn(0x55c40073f4f0 msgr2=0x55c40073f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.069+0000 7f3a345e6740 1 -- 172.21.15.6:0/1289927813 shutdown_connections 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.069+0000 7f3a345e6740 1 -- 172.21.15.6:0/1289927813 wait complete. 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.069+0000 7f3a345e6740 1 -- start start 2024-02-26T06:29:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.069+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f3a100a22c0 0x7f3a100a4780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c005b50 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.069+0000 7f3a345e6740 1 Processor -- start 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c40074b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c40074e2c0 0x55c40074bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c400651120 con 0x55c40074e2c0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c400650dc0 con 0x55c4007456e0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40041a1c0 con 0x55c400744dc0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a345e6740 10 client.0 _ll_register_callbacks cb 0x55c4006a0280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60668/0 (socket says 172.21.15.6:60668) 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 -- 172.21.15.6:0/941759691 learned_addr learned my addr 172.21.15.6:0/941759691 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c40074b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2b400640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c40074e2c0 0x55c40074bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 -- 172.21.15.6:0/941759691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 msgr2=0x55c40074b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c40074b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 -- 172.21.15.6:0/941759691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c40074e2c0 msgr2=0x55c40074bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c40074e2c0 0x55c40074bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2a000640 1 -- 172.21.15.6:0/941759691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a24009bd0 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.070+0000 7f3a2b400640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c40074e2c0 0x55c40074bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a2a000640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074b7d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009610 tx=0x7f3a180963c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac2b57b7f2a6b24 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a0be00640 1 -- 172.21.15.6:0/941759691 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180ba2b0 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a0be00640 1 -- 172.21.15.6:0/941759691 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f3a180850f0 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4003e8720 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a0be00640 1 -- 172.21.15.6:0/941759691 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1809a080 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a345e6740 10 client.25793 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4006aa570 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a0be00640 1 -- 172.21.15.6:0/941759691 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c4003e8720 con 0x55c4007456e0 2024-02-26T06:29:01.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.071+0000 7f3a0be00640 1 -- 172.21.15.6:0/941759691 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3a180cb850 con 0x55c4007456e0 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a345e6740 10 client.25793 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a345e6740 10 client.25793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a345e6740 20 client.25793 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a345e6740 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c40075e320 0x55c400760740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a345e6740 4 client.25793 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c400437890 con 0x55c40075e320 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.072+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c40075e320 0x55c400760740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.073+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c40075e320 0x55c400760740 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8623bf4f1166dbb server_cookie=94944374d162c2c5 in_seq=0 out_seq=0 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.073+0000 7f3a0be00640 10 client.25793 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.073+0000 7f3a0be00640 1 -- 172.21.15.6:0/941759691 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c400437890 con 0x55c40075e320 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.073+0000 7f3a0be00640 10 client.25793 handle_command_reply: tid=0 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.073+0000 7f3a345e6740 1 client.25793 shutdown 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 msgr2=0x55c40074b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074b7d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009610 tx=0x7f3a180963c0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 shutdown_connections 2024-02-26T06:29:01.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c40075e320 0x55c400760740 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c40074e2c0 0x55c40074bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4007456e0 0x55c40074b7d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a0be00640 0 client.25793 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 --2- 172.21.15.6:0/941759691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c40074b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 >> 172.21.15.6:0/941759691 conn(0x55c40073f4f0 msgr2=0x55c400741240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 shutdown_connections 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 1 -- 172.21.15.6:0/941759691 wait complete. 2024-02-26T06:29:01.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.074+0000 7f3a345e6740 20 client.25793 trim_cache size 0 max 16384 2024-02-26T06:29:01.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.088+0000 7f3a345e6740 1 Processor -- start 2024-02-26T06:29:01.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.088+0000 7f3a345e6740 1 -- start start 2024-02-26T06:29:01.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 0x55c40074d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c40074bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a345e6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c400746e90 con 0x55c4007456e0 2024-02-26T06:29:01.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a345e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40073ab90 con 0x55c400744dc0 2024-02-26T06:29:01.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a345e6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40073adc0 con 0x55c40074e2c0 2024-02-26T06:29:01.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 0x55c40074d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 0x55c40074d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60682/0 (socket says 172.21.15.6:60682) 2024-02-26T06:29:01.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 -- 172.21.15.6:0/1730799880 learned_addr learned my addr 172.21.15.6:0/1730799880 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2b400640 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2a000640 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c40074bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 -- 172.21.15.6:0/1730799880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 msgr2=0x55c4007484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 -- 172.21.15.6:0/1730799880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 msgr2=0x55c40074bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c40074bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2aa00640 1 -- 172.21.15.6:0/1730799880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c400646a80 con 0x55c400744dc0 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.089+0000 7f3a2a000640 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c40074bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.090+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 0x55c40074d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc85660841c9ad1e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.090+0000 7f3a16a00640 1 -- 172.21.15.6:0/1730799880 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c052040 con 0x55c400744dc0 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.090+0000 7f3a16a00640 1 -- 172.21.15.6:0/1730799880 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c044070 con 0x55c400744dc0 2024-02-26T06:29:01.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.090+0000 7f3a16a00640 1 -- 172.21.15.6:0/1730799880 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c054640 con 0x55c400744dc0 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 -- 172.21.15.6:0/1730799880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 msgr2=0x55c40074d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 0x55c40074d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c046000 comp rx=0 tx=0).stop 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 -- 172.21.15.6:0/1730799880 shutdown_connections 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c40074e2c0 0x55c4007484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c40074bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 --2- 172.21.15.6:0/1730799880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c400744dc0 0x55c40074d9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 -- 172.21.15.6:0/1730799880 >> 172.21.15.6:0/1730799880 conn(0x55c40073f4f0 msgr2=0x55c4007650a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 -- 172.21.15.6:0/1730799880 shutdown_connections 2024-02-26T06:29:01.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 -- 172.21.15.6:0/1730799880 wait complete. 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 -- start start 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.091+0000 7f3a345e6740 1 Processor -- start 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c4007547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c400754cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 0x55c4007ad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4007ade80 con 0x55c4007456e0 2024-02-26T06:29:01.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4007adff0 con 0x55c40074e2c0 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4007ae1b0 con 0x55c400744dc0 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a345e6740 10 client.0 _ll_register_callbacks cb 0x7f3a24009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c400754cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 0x55c4007ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 0x55c4007ad940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60696/0 (socket says 172.21.15.6:60696) 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 -- 172.21.15.6:0/963990197 learned_addr learned my addr 172.21.15.6:0/963990197 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c4007547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 -- 172.21.15.6:0/963990197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 msgr2=0x55c4007547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c4007547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 -- 172.21.15.6:0/963990197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 msgr2=0x55c400754cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c400754cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.092+0000 7f3a2b400640 1 -- 172.21.15.6:0/963990197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c030070 con 0x55c40074e2c0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a2b400640 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 0x55c4007ad940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009f70 tx=0x7f3a24004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79a0bdd869ff5f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a0be00640 1 -- 172.21.15.6:0/963990197 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24021040 con 0x55c40074e2c0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a0be00640 1 -- 172.21.15.6:0/963990197 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f3a24005550 con 0x55c40074e2c0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4003ab7e0 con 0x55c40074e2c0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a0be00640 1 -- 172.21.15.6:0/963990197 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a24005710 con 0x55c40074e2c0 2024-02-26T06:29:01.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a345e6740 10 client.25799 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.093+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c40066c620 con 0x55c40074e2c0 2024-02-26T06:29:01.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a0be00640 1 -- 172.21.15.6:0/963990197 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c4003ab7e0 con 0x55c40074e2c0 2024-02-26T06:29:01.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a0be00640 1 -- 172.21.15.6:0/963990197 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3a24024070 con 0x55c40074e2c0 2024-02-26T06:29:01.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a345e6740 10 client.25799 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a345e6740 10 client.25799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-02-26T06:29:01.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a345e6740 20 client.25799 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a345e6740 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c4007b6c70 0x55c4007b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a345e6740 4 client.25799 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:01.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c40039a4e0 con 0x55c4007b6c70 2024-02-26T06:29:01.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.094+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c4007b6c70 0x55c4007b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a2aa00640 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c4007b6c70 0x55c4007b9090 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=84be5b1dc7a8c2bb server_cookie=6abe604b464fec6a in_seq=0 out_seq=0 2024-02-26T06:29:01.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a0be00640 10 client.25799 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:01.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a0be00640 1 -- 172.21.15.6:0/963990197 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c40039a4e0 con 0x55c4007b6c70 2024-02-26T06:29:01.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a0be00640 10 client.25799 handle_command_reply: tid=0 2024-02-26T06:29:01.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 client.25799 shutdown 2024-02-26T06:29:01.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 msgr2=0x55c4007ad940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 0x55c4007ad940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009f70 tx=0x7f3a24004cf0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 shutdown_connections 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55c4007b6c70 0x55c4007b9090 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40074e2c0 0x55c4007ad940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4007456e0 0x55c400754cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 --2- 172.21.15.6:0/963990197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c400744dc0 0x55c4007547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a0be00640 0 client.25799 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:01.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.095+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 >> 172.21.15.6:0/963990197 conn(0x55c40073f4f0 msgr2=0x55c40073de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.096+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 shutdown_connections 2024-02-26T06:29:01.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.096+0000 7f3a345e6740 1 -- 172.21.15.6:0/963990197 wait complete. 2024-02-26T06:29:01.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.096+0000 7f3a345e6740 20 client.25799 trim_cache size 0 max 16384 2024-02-26T06:29:01.112 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:01.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f3a100a22c0 msgr2=0x7f3a100a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f3a100a22c0 0x7f3a100a4780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c005b50 comp rx=0 tx=0).stop 2024-02-26T06:29:01.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 msgr2=0x7f3a2c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 0x7f3a2c19fb30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800d950 tx=0x7f3a1800de10 comp rx=0 tx=0).stop 2024-02-26T06:29:01.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 shutdown_connections 2024-02-26T06:29:01.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f3a100a22c0 0x7f3a100a4780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a2c10d820 0x7f3a2c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c073e50 0x7f3a2c19fb30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 --2- 172.21.15.6:0/3611649728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c073530 0x7f3a2c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 >> 172.21.15.6:0/3611649728 conn(0x7f3a2c0fca90 msgr2=0x7f3a2c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 shutdown_connections 2024-02-26T06:29:01.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.097+0000 7f3a2be00640 1 -- 172.21.15.6:0/3611649728 wait complete. 2024-02-26T06:29:01.116 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:00 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1516312372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:01.117 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:29:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:29:00.988+0000 7f2362a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:29:01.117 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:29:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:29:00.988+0000 7f2362a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:29:01.117 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:29:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:29:00.988+0000 7f2362a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:29:01.117 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:29:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:29:00.988+0000 7f2362a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:29:01.118 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:29:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:29:00.988+0000 7f2362a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T06:29:01.119 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:01.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:29:01.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:00 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1516312372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.343+0000 7f442d200640 1 -- 172.21.15.6:0/3678947604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 msgr2=0x7f4428114710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.343+0000 7f442d200640 1 --2- 172.21.15.6:0/3678947604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f4428114710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4420009cd0 tx=0x7f442002c0f0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 -- 172.21.15.6:0/3678947604 shutdown_connections 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 --2- 172.21.15.6:0/3678947604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f4428114710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 --2- 172.21.15.6:0/3678947604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44281078a0 0x7f4428107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 --2- 172.21.15.6:0/3678947604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4428109590 0x7f4428109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 -- 172.21.15.6:0/3678947604 >> 172.21.15.6:0/3678947604 conn(0x7f4428077020 msgr2=0x7f4428105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 -- 172.21.15.6:0/3678947604 shutdown_connections 2024-02-26T06:29:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.344+0000 7f442d200640 1 -- 172.21.15.6:0/3678947604 wait complete. 2024-02-26T06:29:01.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.345+0000 7f442d200640 1 Processor -- start 2024-02-26T06:29:01.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.345+0000 7f442d200640 1 -- start start 2024-02-26T06:29:01.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.345+0000 7f442d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44281078a0 0x7f442819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f442d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f442819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f442d200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4428109590 0x7f44281a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f442d200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44281a2da0 con 0x7f4428109590 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f442d200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44281a2f10 con 0x7f4428108240 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f442d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44281a30d0 con 0x7f44281078a0 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f442819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f442819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60706/0 (socket says 172.21.15.6:60706) 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 -- 172.21.15.6:0/3466517370 learned_addr learned my addr 172.21.15.6:0/3466517370 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425e00640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44281078a0 0x7f442819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4426800640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4428109590 0x7f44281a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 -- 172.21.15.6:0/3466517370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44281078a0 msgr2=0x7f442819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44281078a0 0x7f442819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 -- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4428109590 msgr2=0x7f44281a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4428109590 0x7f44281a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 -- 172.21.15.6:0/3466517370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420009900 con 0x7f4428108240 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425e00640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44281078a0 0x7f442819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4426800640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4428109590 0x7f44281a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.346+0000 7f4425400640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f442819e420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f441400d950 tx=0x7f441400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.347+0000 7f441ea00640 1 -- 172.21.15.6:0/3466517370 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414009a30 con 0x7f4428108240 2024-02-26T06:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.347+0000 7f441ea00640 1 -- 172.21.15.6:0/3466517370 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4414009b90 con 0x7f4428108240 2024-02-26T06:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.347+0000 7f441ea00640 1 -- 172.21.15.6:0/3466517370 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414004a00 con 0x7f4428108240 2024-02-26T06:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.347+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44281a33b0 con 0x7f4428108240 2024-02-26T06:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.347+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44281aac30 con 0x7f4428108240 2024-02-26T06:29:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.350+0000 7f442ffdc740 1 Processor -- start 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.350+0000 7f442ffdc740 1 -- start start 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f441ea00640 1 -- 172.21.15.6:0/3466517370 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f4414009d00 con 0x7f4428108240 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494bf530 0x5598494c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494c8110 0x5598494ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f442ffdc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984942d1e0 con 0x5598494bf530 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f442ffdc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984942a710 con 0x5598494c8110 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f442ffdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559849425ef0 con 0x5598494bec10 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f4426800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f4425e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494bf530 0x5598494c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f4426800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39570/0 (socket says 172.21.15.6:39570) 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f4426800640 1 -- 172.21.15.6:0/944687722 learned_addr learned my addr 172.21.15.6:0/944687722 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.351+0000 7f4425400640 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494c8110 0x5598494ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.352+0000 7f4426800640 1 -- 172.21.15.6:0/944687722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494c8110 msgr2=0x5598494ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.352+0000 7f4426800640 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494c8110 0x5598494ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.352+0000 7f4426800640 1 -- 172.21.15.6:0/944687722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494bf530 msgr2=0x5598494c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.352+0000 7f4426800640 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494bf530 0x5598494c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.352+0000 7f4426800640 1 -- 172.21.15.6:0/944687722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598493e7dc0 con 0x5598494bec10 2024-02-26T06:29:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.352+0000 7f4426800640 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494beff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4420004af0 tx=0x7f4420032de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda4587f2ace4732 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f4413400640 1 -- 172.21.15.6:0/944687722 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420032400 con 0x5598494bec10 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f4413400640 1 -- 172.21.15.6:0/944687722 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4420034a90 con 0x5598494bec10 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f4413400640 1 -- 172.21.15.6:0/944687722 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420003970 con 0x5598494bec10 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f441ea00640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f440c0a2340 0x7f440c0a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f441ea00640 1 -- 172.21.15.6:0/3466517370 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f44140cdcb0 con 0x7f4428108240 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f4425e00640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f440c0a2340 0x7f440c0a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f442ffdc740 1 -- 172.21.15.6:0/944687722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 msgr2=0x5598494beff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.353+0000 7f442ffdc740 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494beff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4420004af0 tx=0x7f4420032de0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 -- 172.21.15.6:0/944687722 shutdown_connections 2024-02-26T06:29:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494c8110 0x5598494ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494bf530 0x5598494c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 --2- 172.21.15.6:0/944687722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494beff0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 -- 172.21.15.6:0/944687722 >> 172.21.15.6:0/944687722 conn(0x5598494b9340 msgr2=0x5598494b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 -- 172.21.15.6:0/944687722 shutdown_connections 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 -- 172.21.15.6:0/944687722 wait complete. 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f4425e00640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f440c0a2340 0x7f440c0a4800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5598494be6e0 tx=0x7f4418005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.354+0000 7f442ffdc740 1 -- start start 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 Processor -- start 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 0x5598494c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bf530 0x5598494c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598493cb120 con 0x5598494c8110 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598493cadc0 con 0x5598494bec10 2024-02-26T06:29:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598491941c0 con 0x5598494bf530 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f442ffdc740 10 client.0 _ll_register_callbacks cb 0x55984941a280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f4425e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 0x5598494c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f4425e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 0x5598494c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60726/0 (socket says 172.21.15.6:60726) 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f4425e00640 1 -- 172.21.15.6:0/4245634170 learned_addr learned my addr 172.21.15.6:0/4245634170 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.355+0000 7f4426800640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425400640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bf530 0x5598494c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425e00640 1 -- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bf530 msgr2=0x5598494c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425e00640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bf530 0x5598494c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425e00640 1 -- 172.21.15.6:0/4245634170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 msgr2=0x5598494c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425e00640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425e00640 1 -- 172.21.15.6:0/4245634170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4428114fa0 con 0x5598494bec10 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4426800640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4425e00640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 0x5598494c5200 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f441801f9d0 tx=0x7f4418004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc26dbcb85be039 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4407e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4407e00640 1 -- 172.21.15.6:0/4245634170 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418017070 con 0x5598494bec10 2024-02-26T06:29:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f4407e00640 1 -- 172.21.15.6:0/4245634170 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4418005020 con 0x5598494bec10 2024-02-26T06:29:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.356+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598491257e0 con 0x5598494bec10 2024-02-26T06:29:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f4407e00640 1 -- 172.21.15.6:0/4245634170 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44180051e0 con 0x5598494bec10 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f442ffdc740 10 client.25811 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559849424570 con 0x5598494bec10 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f4407e00640 1 -- 172.21.15.6:0/4245634170 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4418035080 con 0x5598494bec10 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f4407e00640 1 -- 172.21.15.6:0/4245634170 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f441801d740 con 0x5598494bec10 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f442ffdc740 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f442ffdc740 10 client.25811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f442ffdc740 20 client.25811 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.357+0000 7f442ffdc740 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5598494d82a0 0x5598494da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.358+0000 7f442ffdc740 4 client.25811 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.358+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559849162720 con 0x5598494d82a0 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.358+0000 7f4425400640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5598494d82a0 0x5598494da6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.358+0000 7f4425400640 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5598494d82a0 0x5598494da6e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25565b9c7bd83a56 server_cookie=461717b293ce7f97 in_seq=0 out_seq=0 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.358+0000 7f4407e00640 10 client.25811 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.359+0000 7f4407e00640 1 -- 172.21.15.6:0/4245634170 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559849162720 con 0x5598494d82a0 2024-02-26T06:29:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.359+0000 7f4407e00640 10 client.25811 handle_command_reply: tid=0 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.359+0000 7f442ffdc740 1 client.25811 shutdown 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 msgr2=0x5598494c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 0x5598494c5200 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f441801f9d0 tx=0x7f4418004730 comp rx=0 tx=0).stop 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 shutdown_connections 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5598494d82a0 0x5598494da6e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bf530 0x5598494c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 --2- 172.21.15.6:0/4245634170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bec10 0x5598494c5200 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f4407e00640 0 client.25811 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 >> 172.21.15.6:0/4245634170 conn(0x5598494b9340 msgr2=0x5598494bb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 shutdown_connections 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 1 -- 172.21.15.6:0/4245634170 wait complete. 2024-02-26T06:29:01.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.360+0000 7f442ffdc740 20 client.25811 trim_cache size 0 max 16384 2024-02-26T06:29:01.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:00 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1516312372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:01.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.373+0000 7f442ffdc740 1 Processor -- start 2024-02-26T06:29:01.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.373+0000 7f442ffdc740 1 -- start start 2024-02-26T06:29:01.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f442ffdc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598494ba0d0 con 0x5598494c8110 2024-02-26T06:29:01.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f442ffdc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598494d8050 con 0x5598494bf530 2024-02-26T06:29:01.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f442ffdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598494b4c10 con 0x5598494bec10 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60740/0 (socket says 172.21.15.6:60740) 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 -- 172.21.15.6:0/902804638 learned_addr learned my addr 172.21.15.6:0/902804638 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4426800640 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425e00640 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 -- 172.21.15.6:0/902804638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 msgr2=0x5598494c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 -- 172.21.15.6:0/902804638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 msgr2=0x5598494c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 -- 172.21.15.6:0/902804638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598493c0a80 con 0x5598494bf530 2024-02-26T06:29:01.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4426800640 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:01.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.374+0000 7f4425400640 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494c5aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4414005fd0 tx=0x7f44140ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d5995ef53a152ad server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f4412a00640 1 -- 172.21.15.6:0/902804638 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140967b0 con 0x5598494bf530 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f4412a00640 1 -- 172.21.15.6:0/902804638 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f441409a2e0 con 0x5598494bf530 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f4412a00640 1 -- 172.21.15.6:0/902804638 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441409e6f0 con 0x5598494bf530 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 -- 172.21.15.6:0/902804638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 msgr2=0x5598494c5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494c5aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4414005fd0 tx=0x7f44140ba040 comp rx=0 tx=0).stop 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 -- 172.21.15.6:0/902804638 shutdown_connections 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x5598494c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494c5aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 --2- 172.21.15.6:0/902804638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 -- 172.21.15.6:0/902804638 >> 172.21.15.6:0/902804638 conn(0x5598494b9340 msgr2=0x5598494c3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.375+0000 7f442ffdc740 1 -- 172.21.15.6:0/902804638 shutdown_connections 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.376+0000 7f442ffdc740 1 -- 172.21.15.6:0/902804638 wait complete. 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.376+0000 7f442ffdc740 1 -- start start 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.376+0000 7f442ffdc740 1 Processor -- start 2024-02-26T06:29:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x55984952e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984952ed10 con 0x5598494c8110 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984952ee50 con 0x5598494bf530 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984952f010 con 0x5598494bec10 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f442ffdc740 10 client.0 _ll_register_callbacks cb 0x7f4428067b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60746/0 (socket says 172.21.15.6:60746) 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 -- 172.21.15.6:0/1022513596 learned_addr learned my addr 172.21.15.6:0/1022513596 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425e00640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4426800640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x55984952e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 -- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 msgr2=0x5598494ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 -- 172.21.15.6:0/1022513596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 msgr2=0x55984952e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x55984952e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.377+0000 7f4425400640 1 -- 172.21.15.6:0/1022513596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44281aac30 con 0x5598494bf530 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4425400640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494ce690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44140cb9e0 tx=0x7f44140cb260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d419c970824728 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4407e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4407e00640 1 -- 172.21.15.6:0/1022513596 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140b2070 con 0x5598494bf530 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4407e00640 1 -- 172.21.15.6:0/1022513596 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f441409a510 con 0x5598494bf530 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4420034a90 con 0x5598494bf530 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4407e00640 1 -- 172.21.15.6:0/1022513596 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4414099e70 con 0x5598494bf530 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 10 client.25817 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5598493e6620 con 0x5598494bf530 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4407e00640 1 -- 172.21.15.6:0/1022513596 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4420034a90 con 0x5598494bf530 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f4407e00640 1 -- 172.21.15.6:0/1022513596 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4414096b80 con 0x5598494bf530 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 10 client.25817 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 10 client.25817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 20 client.25817 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.378+0000 7f442ffdc740 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x559849530550 0x559849532970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.379+0000 7f442ffdc740 4 client.25817 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:01.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.379+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598491b1890 con 0x559849530550 2024-02-26T06:29:01.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.379+0000 7f4425e00640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x559849530550 0x559849532970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.379+0000 7f4425e00640 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x559849530550 0x559849532970 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bdd5ca75468ea57 server_cookie=447dc36fa75dd995 in_seq=0 out_seq=0 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.379+0000 7f4407e00640 10 client.25817 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f4407e00640 1 -- 172.21.15.6:0/1022513596 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598491b1890 con 0x559849530550 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f4407e00640 10 client.25817 handle_command_reply: tid=0 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 client.25817 shutdown 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 msgr2=0x5598494ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494ce690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44140cb9e0 tx=0x7f44140cb260 comp rx=0 tx=0).stop 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 shutdown_connections 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x559849530550 0x559849532970 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5598494c8110 0x55984952e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5598494bf530 0x5598494ce690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f4407e00640 0 client.25817 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 --2- 172.21.15.6:0/1022513596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598494bec10 0x5598494ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 >> 172.21.15.6:0/1022513596 conn(0x5598494b9340 msgr2=0x5598494b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 shutdown_connections 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 1 -- 172.21.15.6:0/1022513596 wait complete. 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.380+0000 7f442ffdc740 20 client.25817 trim_cache size 0 max 16384 2024-02-26T06:29:01.388 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f440c0a2340 msgr2=0x7f440c0a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f440c0a2340 0x7f440c0a4800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5598494be6e0 tx=0x7f4418005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 msgr2=0x7f442819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f442819e420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f441400d950 tx=0x7f441400de10 comp rx=0 tx=0).stop 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 shutdown_connections 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f440c0a2340 0x7f440c0a4800 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4428109590 0x7f44281a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4428108240 0x7f442819e420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 --2- 172.21.15.6:0/3466517370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44281078a0 0x7f442819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 >> 172.21.15.6:0/3466517370 conn(0x7f4428077020 msgr2=0x7f44280777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 shutdown_connections 2024-02-26T06:29:01.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.381+0000 7f442d200640 1 -- 172.21.15.6:0/3466517370 wait complete. 2024-02-26T06:29:01.401 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:01.401 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:29:01.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 -- 172.21.15.6:0/2156222517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c108480 msgr2=0x7ffb1c110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 --2- 172.21.15.6:0/2156222517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c108480 0x7ffb1c110050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009cd0 tx=0x7ffb1402c0e0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 -- 172.21.15.6:0/2156222517 shutdown_connections 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 --2- 172.21.15.6:0/2156222517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c108480 0x7ffb1c110050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 --2- 172.21.15.6:0/2156222517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb1c103d60 0x7ffb1c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 --2- 172.21.15.6:0/2156222517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 -- 172.21.15.6:0/2156222517 >> 172.21.15.6:0/2156222517 conn(0x7ffb1c0febf0 msgr2=0x7ffb1c101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 -- 172.21.15.6:0/2156222517 shutdown_connections 2024-02-26T06:29:01.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.624+0000 7ffb21600640 1 -- 172.21.15.6:0/2156222517 wait complete. 2024-02-26T06:29:01.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.625+0000 7ffb21600640 1 Processor -- start 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.625+0000 7ffb21600640 1 -- start start 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.625+0000 7ffb21600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.625+0000 7ffb21600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb1c108480 0x7ffb1c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb21600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb1c199670 0x7ffb1c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb21600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c19a060 con 0x7ffb1c1033b0 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb21600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c19a1d0 con 0x7ffb1c108480 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb21600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c19a390 con 0x7ffb1c199670 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb1c108480 0x7ffb1c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c19f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45810/0 (socket says 172.21.15.6:45810) 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 -- 172.21.15.6:0/366704033 learned_addr learned my addr 172.21.15.6:0/366704033 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb20c00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb1c199670 0x7ffb1c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 -- 172.21.15.6:0/366704033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb1c199670 msgr2=0x7ffb1c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb1c199670 0x7ffb1c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 -- 172.21.15.6:0/366704033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb1c108480 msgr2=0x7ffb1c19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb1c108480 0x7ffb1c19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.626+0000 7ffb1be00640 1 -- 172.21.15.6:0/366704033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009490 con 0x7ffb1c1033b0 2024-02-26T06:29:01.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.627+0000 7ffb1be00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c19f470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00ea60 tx=0x7ffb0c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.627+0000 7ffb18c00640 1 -- 172.21.15.6:0/366704033 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00cde0 con 0x7ffb1c1033b0 2024-02-26T06:29:01.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.627+0000 7ffb18c00640 1 -- 172.21.15.6:0/366704033 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb0c0035f0 con 0x7ffb1c1033b0 2024-02-26T06:29:01.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.627+0000 7ffb18c00640 1 -- 172.21.15.6:0/366704033 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c010690 con 0x7ffb1c1033b0 2024-02-26T06:29:01.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.627+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb08009490 con 0x7ffb1c1033b0 2024-02-26T06:29:01.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.627+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14009900 con 0x7ffb1c1033b0 2024-02-26T06:29:01.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.631+0000 7ffb25a06740 1 Processor -- start 2024-02-26T06:29:01.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 -- start start 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb18c00640 1 -- 172.21.15.6:0/366704033 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7ffb0c003af0 con 0x7ffb1c1033b0 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca59d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89ca5c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89c9bf1e0 con 0x55b89ca5a2c0 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89c9bc710 con 0x55b89ca516e0 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb25a06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89c9b7ef0 con 0x55b89ca50dc0 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.632+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca59d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb20c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca59d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60774/0 (socket says 172.21.15.6:60774) 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 -- 172.21.15.6:0/120891121 learned_addr learned my addr 172.21.15.6:0/120891121 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1b400640 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89ca5c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 -- 172.21.15.6:0/120891121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 msgr2=0x55b89ca511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 -- 172.21.15.6:0/120891121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 msgr2=0x55b89ca5c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89ca5c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 -- 172.21.15.6:0/120891121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89c979dc0 con 0x55b89ca516e0 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.633+0000 7ffb1be00640 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca59d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0cbc50 tx=0x7ffb0c0d6880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d3706f75a8df01 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.634+0000 7ffb11200640 1 -- 172.21.15.6:0/120891121 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0cecd0 con 0x55b89ca516e0 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.634+0000 7ffb11200640 1 -- 172.21.15.6:0/120891121 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb0c0cee30 con 0x55b89ca516e0 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.634+0000 7ffb11200640 1 -- 172.21.15.6:0/120891121 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0dab70 con 0x55b89ca516e0 2024-02-26T06:29:01.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.634+0000 7ffb18c00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ffb000a2340 0x7ffb000a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.634+0000 7ffb18c00640 1 -- 172.21.15.6:0/366704033 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7ffb0c014070 con 0x7ffb1c1033b0 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.634+0000 7ffb20c00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ffb000a2340 0x7ffb000a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 -- 172.21.15.6:0/120891121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 msgr2=0x55b89ca59d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca59d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0cbc50 tx=0x7ffb0c0d6880 comp rx=0 tx=0).stop 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 -- 172.21.15.6:0/120891121 shutdown_connections 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89ca5c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca59d80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 --2- 172.21.15.6:0/120891121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 -- 172.21.15.6:0/120891121 >> 172.21.15.6:0/120891121 conn(0x55b89ca4b4f0 msgr2=0x55b89ca4b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 -- 172.21.15.6:0/120891121 shutdown_connections 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb20c00640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ffb000a2340 0x7ffb000a4800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb14005cc0 tx=0x7ffb14005c50 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.635+0000 7ffb25a06740 1 -- 172.21.15.6:0/120891121 wait complete. 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 -- start start 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 Processor -- start 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 0x55b89ca57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca516e0 0x55b89ca57890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca57dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89c95d120 con 0x55b89ca516e0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89c95cdc0 con 0x55b89ca50dc0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89c7261c0 con 0x55b89ca5a2c0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.636+0000 7ffb25a06740 10 client.0 _ll_register_callbacks cb 0x55b89c9ac280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 0x55b89ca57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca516e0 0x55b89ca57890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 0x55b89ca57350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60782/0 (socket says 172.21.15.6:60782) 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 -- 172.21.15.6:0/3984117409 learned_addr learned my addr 172.21.15.6:0/3984117409 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb20c00640 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca57dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 -- 172.21.15.6:0/3984117409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 msgr2=0x55b89ca57dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca57dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 -- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca516e0 msgr2=0x55b89ca57890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca516e0 0x55b89ca57890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 -- 172.21.15.6:0/3984117409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14009900 con 0x55b89ca50dc0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb1be00640 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 0x55b89ca57350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0cbc20 tx=0x7ffb0c0cbc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75398ebe56e552b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb05600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb05600640 1 -- 172.21.15.6:0/3984117409 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0da4f0 con 0x55b89ca50dc0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.637+0000 7ffb05600640 1 -- 172.21.15.6:0/3984117409 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7ffb0c0da300 con 0x55b89ca50dc0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b89c6b77e0 con 0x55b89ca50dc0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb05600640 1 -- 172.21.15.6:0/3984117409 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb0c0d1d50 con 0x55b89ca50dc0 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 10 client.25829 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b89c9b6570 con 0x55b89ca50dc0 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb05600640 1 -- 172.21.15.6:0/3984117409 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b89c6b77e0 con 0x55b89ca50dc0 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb05600640 1 -- 172.21.15.6:0/3984117409 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb0c0cead0 con 0x55b89ca50dc0 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 10 client.25829 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 10 client.25829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 20 client.25829 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89ca69fb0 0x55b89ca6c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 4 client.25829 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.638+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b89c6f4720 con 0x55b89ca69fb0 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.639+0000 7ffb1b400640 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89ca69fb0 0x55b89ca6c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.639+0000 7ffb1b400640 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89ca69fb0 0x55b89ca6c3f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87b1f1d2c830f514 server_cookie=ef382a3263d7d19d in_seq=0 out_seq=0 2024-02-26T06:29:01.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.639+0000 7ffb05600640 10 client.25829 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:01.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb05600640 1 -- 172.21.15.6:0/3984117409 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b89c6f4720 con 0x55b89ca69fb0 2024-02-26T06:29:01.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb05600640 10 client.25829 handle_command_reply: tid=0 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 client.25829 shutdown 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 msgr2=0x55b89ca57350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 0x55b89ca57350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0cbc20 tx=0x7ffb0c0cbc80 comp rx=0 tx=0).stop 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 shutdown_connections 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89ca69fb0 0x55b89ca6c3f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca57dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca516e0 0x55b89ca57890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb05600640 0 client.25829 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3984117409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca50dc0 0x55b89ca57350 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 >> 172.21.15.6:0/3984117409 conn(0x55b89ca4b4f0 msgr2=0x55b89ca4d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 shutdown_connections 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.640+0000 7ffb25a06740 1 -- 172.21.15.6:0/3984117409 wait complete. 2024-02-26T06:29:01.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.641+0000 7ffb25a06740 20 client.25829 trim_cache size 0 max 16384 2024-02-26T06:29:01.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 Processor -- start 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 -- start start 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 0x55b89ca59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca52ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89ca4c4f0 con 0x55b89ca50dc0 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89ca66ea0 con 0x55b89ca516e0 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb25a06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89ca50800 con 0x55b89ca5a2c0 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 0x55b89ca59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.654+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 0x55b89ca59a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45846/0 (socket says 172.21.15.6:45846) 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1be00640 1 -- 172.21.15.6:0/2416891166 learned_addr learned my addr 172.21.15.6:0/2416891166 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca52660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb20c00640 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca52ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 -- 172.21.15.6:0/2416891166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 msgr2=0x55b89ca52ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca52ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 -- 172.21.15.6:0/2416891166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 msgr2=0x55b89ca59a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 0x55b89ca59a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 -- 172.21.15.6:0/2416891166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89c952a80 con 0x55b89ca516e0 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1be00640 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 0x55b89ca59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb1b400640 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca52660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55b89ca4c680 tx=0x7ffb08005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3baf5c5bfcf4871d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.655+0000 7ffb07e00640 1 -- 172.21.15.6:0/2416891166 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08017040 con 0x55b89ca516e0 2024-02-26T06:29:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb07e00640 1 -- 172.21.15.6:0/2416891166 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb08018040 con 0x55b89ca516e0 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb07e00640 1 -- 172.21.15.6:0/2416891166 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08010710 con 0x55b89ca516e0 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 -- 172.21.15.6:0/2416891166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 msgr2=0x55b89ca52660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca52660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55b89ca4c680 tx=0x7ffb08005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 -- 172.21.15.6:0/2416891166 shutdown_connections 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca5a2c0 0x55b89ca52ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca52660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 --2- 172.21.15.6:0/2416891166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca50dc0 0x55b89ca59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 -- 172.21.15.6:0/2416891166 >> 172.21.15.6:0/2416891166 conn(0x55b89ca4b4f0 msgr2=0x55b89ca46520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 -- 172.21.15.6:0/2416891166 shutdown_connections 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.656+0000 7ffb25a06740 1 -- 172.21.15.6:0/2416891166 wait complete. 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.657+0000 7ffb25a06740 1 -- start start 2024-02-26T06:29:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.657+0000 7ffb25a06740 1 Processor -- start 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.657+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca60490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.657+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb25a06740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89caba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb25a06740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89cabaaa0 con 0x55b89ca5a2c0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb25a06740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89cabac10 con 0x55b89ca516e0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb25a06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89cabadd0 con 0x55b89ca50dc0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb25a06740 10 client.0 _ll_register_callbacks cb 0x7ffb0c0cbb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca60490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca60490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39648/0 (socket says 172.21.15.6:39648) 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 -- 172.21.15.6:0/3371897254 learned_addr learned my addr 172.21.15.6:0/3371897254 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb20c00640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89caba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1b400640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 -- 172.21.15.6:0/3371897254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 msgr2=0x55b89ca609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 -- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 msgr2=0x55b89caba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89caba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.658+0000 7ffb1be00640 1 -- 172.21.15.6:0/3371897254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89c9b6570 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb1be00640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca60490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c097330 tx=0x7ffb0c0d6170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1f232243fb406f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb05600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb05600640 1 -- 172.21.15.6:0/3371897254 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0ce750 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb05600640 1 -- 172.21.15.6:0/3371897254 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7ffb0c09b5d0 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb0c0cee30 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb05600640 1 -- 172.21.15.6:0/3371897254 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb0c0e0070 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb25a06740 10 client.25816 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ffb08009850 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.659+0000 7ffb05600640 1 -- 172.21.15.6:0/3371897254 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb0c0cee30 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb05600640 1 -- 172.21.15.6:0/3371897254 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb0c0b9280 con 0x55b89ca50dc0 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb25a06740 10 client.25816 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb25a06740 10 client.25816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb25a06740 20 client.25816 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89cac2240 0x55b89cac4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb25a06740 4 client.25816 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b89c743890 con 0x55b89cac2240 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.660+0000 7ffb1b400640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89cac2240 0x55b89cac4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb05600640 10 client.25816 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb1b400640 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89cac2240 0x55b89cac4630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f8dbe83077879a9 server_cookie=65c2ce7599f77f17 in_seq=0 out_seq=0 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb05600640 1 -- 172.21.15.6:0/3371897254 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b89c743890 con 0x55b89cac2240 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb05600640 10 client.25816 handle_command_reply: tid=0 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 client.25816 shutdown 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 msgr2=0x55b89ca60490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca60490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c097330 tx=0x7ffb0c0d6170 comp rx=0 tx=0).stop 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 shutdown_connections 2024-02-26T06:29:01.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x55b89cac2240 0x55b89cac4630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b89ca5a2c0 0x55b89caba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b89ca516e0 0x55b89ca609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 --2- 172.21.15.6:0/3371897254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b89ca50dc0 0x55b89ca60490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb05600640 0 client.25816 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 >> 172.21.15.6:0/3371897254 conn(0x55b89ca4b4f0 msgr2=0x55b89ca52660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 shutdown_connections 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 1 -- 172.21.15.6:0/3371897254 wait complete. 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.661+0000 7ffb25a06740 20 client.25816 trim_cache size 0 max 16384 2024-02-26T06:29:01.665 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ffb000a2340 msgr2=0x7ffb000a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ffb000a2340 0x7ffb000a4800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb14005cc0 tx=0x7ffb14005c50 comp rx=0 tx=0).stop 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 msgr2=0x7ffb1c19f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c19f470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00ea60 tx=0x7ffb0c00ef20 comp rx=0 tx=0).stop 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 shutdown_connections 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ffb000a2340 0x7ffb000a4800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb1c199670 0x7ffb1c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb1c108480 0x7ffb1c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 --2- 172.21.15.6:0/366704033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb1c1033b0 0x7ffb1c19f470 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 >> 172.21.15.6:0/366704033 conn(0x7ffb1c0febf0 msgr2=0x7ffb1c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 shutdown_connections 2024-02-26T06:29:01.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.662+0000 7ffb21600640 1 -- 172.21.15.6:0/366704033 wait complete. 2024-02-26T06:29:01.680 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:01.680 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:29:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.886+0000 7f4300000640 1 -- 172.21.15.6:0/2122916148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101d30 msgr2=0x7f42f8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.886+0000 7f4300000640 1 --2- 172.21.15.6:0/2122916148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101d30 0x7f42f8105d70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009930 tx=0x7f42e402bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 -- 172.21.15.6:0/2122916148 shutdown_connections 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 --2- 172.21.15.6:0/2122916148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f8106450 0x7f42f810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 --2- 172.21.15.6:0/2122916148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101d30 0x7f42f8105d70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 --2- 172.21.15.6:0/2122916148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f8101380 0x7f42f8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 -- 172.21.15.6:0/2122916148 >> 172.21.15.6:0/2122916148 conn(0x7f42f80fcc20 msgr2=0x7f42f80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 -- 172.21.15.6:0/2122916148 shutdown_connections 2024-02-26T06:29:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.887+0000 7f4300000640 1 -- 172.21.15.6:0/2122916148 wait complete. 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 Processor -- start 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 -- start start 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101380 0x7f42f819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f8106450 0x7f42f819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f81998f0 0x7f42f8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f819a2e0 con 0x7f42f8101380 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f819a450 con 0x7f42f81998f0 2024-02-26T06:29:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f4300000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f819a610 con 0x7f42f8106450 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.888+0000 7f42fd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101380 0x7f42f819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f81998f0 0x7f42f8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101380 0x7f42f819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45882/0 (socket says 172.21.15.6:45882) 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fd600640 1 -- 172.21.15.6:0/1007055827 learned_addr learned my addr 172.21.15.6:0/1007055827 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fcc00640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f8106450 0x7f42f819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 -- 172.21.15.6:0/1007055827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f8106450 msgr2=0x7f42f819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f8106450 0x7f42f819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 -- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101380 msgr2=0x7f42f819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101380 0x7f42f819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 -- 172.21.15.6:0/1007055827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4009560 con 0x7f42f81998f0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.889+0000 7f42fe000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f81998f0 0x7f42f8199da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009fd0 tx=0x7f42f000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.890+0000 7f42f6000640 1 -- 172.21.15.6:0/1007055827 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000ce20 con 0x7f42f81998f0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.890+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f81a65b0 con 0x7f42f81998f0 2024-02-26T06:29:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.890+0000 7f42f6000640 1 -- 172.21.15.6:0/1007055827 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f0005020 con 0x7f42f81998f0 2024-02-26T06:29:01.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.890+0000 7f42f6000640 1 -- 172.21.15.6:0/1007055827 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0010690 con 0x7f42f81998f0 2024-02-26T06:29:01.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.890+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f81a6a90 con 0x7f42f81998f0 2024-02-26T06:29:01.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f42f6000640 1 -- 172.21.15.6:0/1007055827 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f42f0005500 con 0x7f42f81998f0 2024-02-26T06:29:01.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 Processor -- start 2024-02-26T06:29:01.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 -- start start 2024-02-26T06:29:01.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9debff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9df4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9d5a1e0 con 0x5634c9debc10 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9d57710 con 0x5634c9df5110 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.893+0000 7f4302f9f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9d52ef0 con 0x5634c9dec530 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9debff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9debff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45884/0 (socket says 172.21.15.6:45884) 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 -- 172.21.15.6:0/2652359205 learned_addr learned my addr 172.21.15.6:0/2652359205 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fd600640 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9df4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 -- 172.21.15.6:0/2652359205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 msgr2=0x5634c9df4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9df4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 -- 172.21.15.6:0/2652359205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 msgr2=0x5634c9df7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fcc00640 1 -- 172.21.15.6:0/2652359205 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634c9d14dc0 con 0x5634c9debc10 2024-02-26T06:29:01.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42fd600640 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9df4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:01.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.894+0000 7f42f6000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f42e00a22c0 0x7f42e00a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.895+0000 7f42fcc00640 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9debff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009900 tx=0x7f42e4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0fe6b815ac9c70 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.895+0000 7f42f6000640 1 -- 172.21.15.6:0/1007055827 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f42f0014070 con 0x7f42f81998f0 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.895+0000 7f42fd600640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f42e00a22c0 0x7f42e00a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.895+0000 7f42ea600640 1 -- 172.21.15.6:0/2652359205 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004a80 con 0x5634c9debc10 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.895+0000 7f42ea600640 1 -- 172.21.15.6:0/2652359205 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e4004030 con 0x5634c9debc10 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.895+0000 7f42ea600640 1 -- 172.21.15.6:0/2652359205 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4005310 con 0x5634c9debc10 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 -- 172.21.15.6:0/2652359205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 msgr2=0x5634c9debff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9debff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009900 tx=0x7f42e4005fd0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f42fd600640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f42e00a22c0 0x7f42e00a4780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009640 tx=0x7f42ec005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 -- 172.21.15.6:0/2652359205 shutdown_connections 2024-02-26T06:29:01.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9df4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2652359205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9debff0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 -- 172.21.15.6:0/2652359205 >> 172.21.15.6:0/2652359205 conn(0x5634c9de6340 msgr2=0x5634c9de6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 -- 172.21.15.6:0/2652359205 shutdown_connections 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.896+0000 7f4302f9f740 1 -- 172.21.15.6:0/2652359205 wait complete. 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 -- start start 2024-02-26T06:29:01.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 Processor -- start 2024-02-26T06:29:01.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9debc10 0x5634c9df20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9dec530 0x5634c9df2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9cf8120 con 0x5634c9dec530 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9cf7dc0 con 0x5634c9df5110 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9ac11c0 con 0x5634c9debc10 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.897+0000 7f4302f9f740 10 client.0 _ll_register_callbacks cb 0x5634c9d47280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fcc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9dec530 0x5634c9df2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9debc10 0x5634c9df20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60848/0 (socket says 172.21.15.6:60848) 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 -- 172.21.15.6:0/2394302458 learned_addr learned my addr 172.21.15.6:0/2394302458 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 -- 172.21.15.6:0/2394302458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9debc10 msgr2=0x5634c9df20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9debc10 0x5634c9df20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 -- 172.21.15.6:0/2394302458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9dec530 msgr2=0x5634c9df2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9dec530 0x5634c9df2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 -- 172.21.15.6:0/2394302458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f810e830 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42fe000640 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df2b70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f42f009aef0 tx=0x7f42f00052c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d768679cb136b7d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.898+0000 7f42dea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f42dea00640 1 -- 172.21.15.6:0/2394302458 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00a3700 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f42dea00640 1 -- 172.21.15.6:0/2394302458 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f42f009f340 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634c9a527e0 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f42dea00640 1 -- 172.21.15.6:0/2394302458 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42f009f170 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 10 client.25847 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5634c9d51570 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f42dea00640 1 -- 172.21.15.6:0/2394302458 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5634c9a527e0 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f42dea00640 1 -- 172.21.15.6:0/2394302458 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f42f001f070 con 0x5634c9df5110 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 10 client.25847 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 10 client.25847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:01.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 20 client.25847 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e051c0 0x5634c9e075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 4 client.25847 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:01.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.899+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634c9a8f720 con 0x5634c9e051c0 2024-02-26T06:29:01.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.900+0000 7f42fd600640 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e051c0 0x5634c9e075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.900+0000 7f42fd600640 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e051c0 0x5634c9e075b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7a5d4a031d2b76 server_cookie=e0b8350eb8697a55 in_seq=0 out_seq=0 2024-02-26T06:29:01.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.900+0000 7f42dea00640 10 client.25847 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f42dea00640 1 -- 172.21.15.6:0/2394302458 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634c9a8f720 con 0x5634c9e051c0 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f42dea00640 10 client.25847 handle_command_reply: tid=0 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 client.25847 shutdown 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 msgr2=0x5634c9df2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df2b70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f42f009aef0 tx=0x7f42f00052c0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 shutdown_connections 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e051c0 0x5634c9e075b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9df2b70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9dec530 0x5634c9df2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f42dea00640 0 client.25847 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 --2- 172.21.15.6:0/2394302458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9debc10 0x5634c9df20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.901+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 >> 172.21.15.6:0/2394302458 conn(0x5634c9de6340 msgr2=0x5634c9de8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.902+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 shutdown_connections 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.902+0000 7f4302f9f740 1 -- 172.21.15.6:0/2394302458 wait complete. 2024-02-26T06:29:01.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.902+0000 7f4302f9f740 20 client.25847 trim_cache size 0 max 16384 2024-02-26T06:29:01.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 Processor -- start 2024-02-26T06:29:01.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 -- start start 2024-02-26T06:29:01.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9df4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9dec530 0x5634c9df2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9df5110 0x5634c9ded450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9de1ab0 con 0x5634c9debc10 2024-02-26T06:29:01.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9e00f00 con 0x5634c9dec530 2024-02-26T06:29:01.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.915+0000 7f4302f9f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9e0c840 con 0x5634c9df5110 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9df4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fe000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9df5110 0x5634c9ded450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9df4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45908/0 (socket says 172.21.15.6:45908) 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 -- 172.21.15.6:0/401641137 learned_addr learned my addr 172.21.15.6:0/401641137 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fcc00640 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9dec530 0x5634c9df2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 -- 172.21.15.6:0/401641137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9df5110 msgr2=0x5634c9ded450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9df5110 0x5634c9ded450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 -- 172.21.15.6:0/401641137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9dec530 msgr2=0x5634c9df2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9dec530 0x5634c9df2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 -- 172.21.15.6:0/401641137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634c9ceda80 con 0x5634c9debc10 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fe000640 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9df5110 0x5634c9ded450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fcc00640 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9dec530 0x5634c9df2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.916+0000 7f42fd600640 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9df4820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5634c9e07e90 tx=0x7f42ec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca85f20b15dbb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f42e9c00640 1 -- 172.21.15.6:0/401641137 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec023030 con 0x5634c9debc10 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f42e9c00640 1 -- 172.21.15.6:0/401641137 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42ec024040 con 0x5634c9debc10 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f42e9c00640 1 -- 172.21.15.6:0/401641137 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec017070 con 0x5634c9debc10 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 -- 172.21.15.6:0/401641137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 msgr2=0x5634c9df4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9df4820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5634c9e07e90 tx=0x7f42ec009f90 comp rx=0 tx=0).stop 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 -- 172.21.15.6:0/401641137 shutdown_connections 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9df5110 0x5634c9ded450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9dec530 0x5634c9df2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 --2- 172.21.15.6:0/401641137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9df4820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.917+0000 7f4302f9f740 1 -- 172.21.15.6:0/401641137 >> 172.21.15.6:0/401641137 conn(0x5634c9de6340 msgr2=0x5634c9dec910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.918+0000 7f4302f9f740 1 -- 172.21.15.6:0/401641137 shutdown_connections 2024-02-26T06:29:01.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.918+0000 7f4302f9f740 1 -- 172.21.15.6:0/401641137 wait complete. 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.918+0000 7f4302f9f740 1 -- start start 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.918+0000 7f4302f9f740 1 Processor -- start 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9dfc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9dfa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9dfadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9dfb4c0 con 0x5634c9debc10 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9dfb630 con 0x5634c9df5110 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c9e5b6d0 con 0x5634c9dec530 2024-02-26T06:29:01.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f4302f9f740 10 client.0 _ll_register_callbacks cb 0x7f42ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9dfc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9dfc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45924/0 (socket says 172.21.15.6:45924) 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 -- 172.21.15.6:0/324677998 learned_addr learned my addr 172.21.15.6:0/324677998 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fcc00640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9dfa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fe000640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9dfadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 -- 172.21.15.6:0/324677998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 msgr2=0x5634c9dfa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9dfa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 -- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 msgr2=0x5634c9dfadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.919+0000 7f42fd600640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9dfadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f42fd600640 1 -- 172.21.15.6:0/324677998 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634c9d14dc0 con 0x5634c9debc10 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f42fd600640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9dfc3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00f630 tx=0x7f42ec01d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588d1563cabbbc56 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f42dea00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f42dea00640 1 -- 172.21.15.6:0/324677998 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec017050 con 0x5634c9debc10 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42e4004030 con 0x5634c9debc10 2024-02-26T06:29:01.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f42dea00640 1 -- 172.21.15.6:0/324677998 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f42ec01da90 con 0x5634c9debc10 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f42dea00640 1 -- 172.21.15.6:0/324677998 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ec023040 con 0x5634c9debc10 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.920+0000 7f4302f9f740 10 client.15708 fetch_fsmap learned FSMap version 27 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5634c9d51570 con 0x5634c9debc10 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f42dea00640 1 -- 172.21.15.6:0/324677998 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e4004030 con 0x5634c9debc10 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f42dea00640 1 -- 172.21.15.6:0/324677998 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f42ec005440 con 0x5634c9debc10 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 20 client.15708 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e5cea0 0x5634c9e5f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 4 client.15708 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634c9ade890 con 0x5634c9e5cea0 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.921+0000 7f42fcc00640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e5cea0 0x5634c9e5f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.922+0000 7f42dea00640 10 client.15708 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:01.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.922+0000 7f42fcc00640 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e5cea0 0x5634c9e5f2c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76be64cdc0a04f98 server_cookie=1ae70c940239bc07 in_seq=0 out_seq=0 2024-02-26T06:29:01.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.922+0000 7f42dea00640 1 -- 172.21.15.6:0/324677998 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634c9ade890 con 0x5634c9e5cea0 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.922+0000 7f42dea00640 10 client.15708 handle_command_reply: tid=0 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.922+0000 7f4302f9f740 1 client.15708 shutdown 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 msgr2=0x5634c9dfc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9dfc3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00f630 tx=0x7f42ec01d8f0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 shutdown_connections 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5634c9e5cea0 0x5634c9e5f2c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c9df5110 0x5634c9dfadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634c9dec530 0x5634c9dfa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 --2- 172.21.15.6:0/324677998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634c9debc10 0x5634c9dfc3d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f42dea00640 0 client.15708 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 >> 172.21.15.6:0/324677998 conn(0x5634c9de6340 msgr2=0x5634c9df05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 shutdown_connections 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 1 -- 172.21.15.6:0/324677998 wait complete. 2024-02-26T06:29:01.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.923+0000 7f4302f9f740 20 client.15708 trim_cache size 0 max 16384 2024-02-26T06:29:01.927 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:01.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f42e00a22c0 msgr2=0x7f42e00a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f42e00a22c0 0x7f42e00a4780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009640 tx=0x7f42ec005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f81998f0 msgr2=0x7f42f8199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f81998f0 0x7f42f8199da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009fd0 tx=0x7f42f000ef10 comp rx=0 tx=0).stop 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 shutdown_connections 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f42e00a22c0 0x7f42e00a4780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42f81998f0 0x7f42f8199da0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f8106450 0x7f42f819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 --2- 172.21.15.6:0/1007055827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42f8101380 0x7f42f819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 >> 172.21.15.6:0/1007055827 conn(0x7f42f80fcc20 msgr2=0x7f42f80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 shutdown_connections 2024-02-26T06:29:01.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:01.924+0000 7f4300000640 1 -- 172.21.15.6:0/1007055827 wait complete. 2024-02-26T06:29:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 -- 172.21.15.6:0/2703883923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781035d0 msgr2=0x7f46781039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 --2- 172.21.15.6:0/2703883923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781035d0 0x7f46781039b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4668009930 tx=0x7f466802bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 -- 172.21.15.6:0/2703883923 shutdown_connections 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 --2- 172.21.15.6:0/2703883923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46781086a0 0x7f4678110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 --2- 172.21.15.6:0/2703883923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4678103f80 0x7f4678107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 --2- 172.21.15.6:0/2703883923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781035d0 0x7f46781039b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 -- 172.21.15.6:0/2703883923 >> 172.21.15.6:0/2703883923 conn(0x7f46780fee70 msgr2=0x7f4678101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 -- 172.21.15.6:0/2703883923 shutdown_connections 2024-02-26T06:29:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.019+0000 7f467ec00640 1 -- 172.21.15.6:0/2703883923 wait complete. 2024-02-26T06:29:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.020+0000 7f467ec00640 1 Processor -- start 2024-02-26T06:29:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.020+0000 7f467ec00640 1 -- start start 2024-02-26T06:29:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 0x7f467819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467ec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 0x7f467819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467ec00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46781998b0 0x7f4678199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467ec00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467819a2a0 con 0x7f46781998b0 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467ec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467819a410 con 0x7f46781086a0 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467819a5d0 con 0x7f46781035d0 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f4677400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 0x7f467819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f4677e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 0x7f467819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f4677400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 0x7f467819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60886/0 (socket says 172.21.15.6:60886) 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f4677400640 1 -- 172.21.15.6:0/1132042059 learned_addr learned my addr 172.21.15.6:0/1132042059 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f4677e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 0x7f467819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39736/0 (socket says 172.21.15.6:39736) 2024-02-26T06:29:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.021+0000 7f467cc00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46781998b0 0x7f4678199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677400640 1 -- 172.21.15.6:0/1132042059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 msgr2=0x7f467819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677400640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 0x7f467819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677400640 1 -- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46781998b0 msgr2=0x7f4678199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677400640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46781998b0 0x7f4678199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677400640 1 -- 172.21.15.6:0/1132042059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4668009560 con 0x7f46781086a0 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677e00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 0x7f467819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4677400640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 0x7f467819fbf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f466400e9f0 tx=0x7f466400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4674c00640 1 -- 172.21.15.6:0/1132042059 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46640096c0 con 0x7f46781086a0 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46781a6570 con 0x7f46781086a0 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4674c00640 1 -- 172.21.15.6:0/1132042059 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4664005500 con 0x7f46781086a0 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.022+0000 7f4674c00640 1 -- 172.21.15.6:0/1132042059 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664010430 con 0x7f46781086a0 2024-02-26T06:29:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.023+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46781a6a00 con 0x7f46781086a0 2024-02-26T06:29:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.026+0000 7f4674c00640 1 -- 172.21.15.6:0/1132042059 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 136943+0+0 (secure 0 0 0) 0x7f4664005020 con 0x7f46781086a0 2024-02-26T06:29:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 Processor -- start 2024-02-26T06:29:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 -- start start 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7937af0 0x5559c7936840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7937200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c79426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c78f25f0 con 0x5559c7937af0 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c78a4890 con 0x5559c7936d80 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4681c1c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c78a1dc0 con 0x5559c7940300 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f467cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7937af0 0x5559c7936840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4677400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c79426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f467cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7937af0 0x5559c7936840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45954/0 (socket says 172.21.15.6:45954) 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f467cc00640 1 -- 172.21.15.6:0/1002249609 learned_addr learned my addr 172.21.15.6:0/1002249609 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.027+0000 7f4677e00640 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7937200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4674c00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f465c0a22b0 0x7f465c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4677400640 1 -- 172.21.15.6:0/1002249609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 msgr2=0x5559c7937200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4677400640 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7937200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4677400640 1 -- 172.21.15.6:0/1002249609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7937af0 msgr2=0x5559c7936840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4677400640 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7937af0 0x5559c7936840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4677400640 1 -- 172.21.15.6:0/1002249609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c781ead0 con 0x5559c7940300 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4674c00640 1 -- 172.21.15.6:0/1132042059 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4664014070 con 0x7f46781086a0 2024-02-26T06:29:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f467cc00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f465c0a22b0 0x7f465c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.028+0000 7f4677400640 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c79426f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46640cbc10 tx=0x7f4664021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97accf1a45f0d449 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.029+0000 7f466d200640 1 -- 172.21.15.6:0/1002249609 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466409abd0 con 0x5559c7940300 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.029+0000 7f466d200640 1 -- 172.21.15.6:0/1002249609 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f466409a940 con 0x5559c7940300 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.029+0000 7f466d200640 1 -- 172.21.15.6:0/1002249609 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466409ae00 con 0x5559c7940300 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 -- 172.21.15.6:0/1002249609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 msgr2=0x5559c79426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c79426f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46640cbc10 tx=0x7f4664021000 comp rx=0 tx=0).stop 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f467cc00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f465c0a22b0 0x7f465c0a4770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4670010610 tx=0x7f4670004a40 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 -- 172.21.15.6:0/1002249609 shutdown_connections 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c79426f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7937200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1002249609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7937af0 0x5559c7936840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 -- 172.21.15.6:0/1002249609 >> 172.21.15.6:0/1002249609 conn(0x5559c78fe480 msgr2=0x5559c7931a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 -- 172.21.15.6:0/1002249609 shutdown_connections 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 -- 172.21.15.6:0/1002249609 wait complete. 2024-02-26T06:29:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 -- start start 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.030+0000 7f4681c1c740 1 Processor -- start 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 0x5559c793d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c793dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c789d5a0 con 0x5559c7940300 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c7842b20 con 0x5559c7937af0 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c78427c0 con 0x5559c7936d80 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4681c1c740 10 client.0 _ll_register_callbacks cb 0x5559c7856520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4677e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 0x5559c793d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4677e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 0x5559c793d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39760/0 (socket says 172.21.15.6:39760) 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4677e00640 1 -- 172.21.15.6:0/321728625 learned_addr learned my addr 172.21.15.6:0/321728625 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f467cc00640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c793dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.031+0000 7f4677400640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.032+0000 7f4677e00640 1 -- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 msgr2=0x5559c793d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.032+0000 7f4677e00640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.032+0000 7f4677e00640 1 -- 172.21.15.6:0/321728625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 msgr2=0x5559c793dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.032+0000 7f4677e00640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c793dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.032+0000 7f4677e00640 1 -- 172.21.15.6:0/321728625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46781a6a00 con 0x5559c7936d80 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.033+0000 7f4677e00640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 0x5559c793d2b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4668000c00 tx=0x7f466802c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f7e1b52c2701302 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.033+0000 7f4661600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.034+0000 7f4661600640 1 -- 172.21.15.6:0/321728625 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668004410 con 0x5559c7936d80 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.034+0000 7f4661600640 1 -- 172.21.15.6:0/321728625 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4668046070 con 0x5559c7936d80 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.034+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559c786fca0 con 0x5559c7936d80 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4661600640 1 -- 172.21.15.6:0/321728625 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4668004df0 con 0x5559c7936d80 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 10 client.25846 fetch_fsmap learned FSMap version 28 2024-02-26T06:29:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5559c789bc20 con 0x5559c7936d80 2024-02-26T06:29:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4661600640 1 -- 172.21.15.6:0/321728625 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5559c786fca0 con 0x5559c7936d80 2024-02-26T06:29:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4661600640 1 -- 172.21.15.6:0/321728625 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f466803c040 con 0x5559c7936d80 2024-02-26T06:29:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 10 client.25846 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:29:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 20 client.25846 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c7950220 0x5559c7952640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 4 client.25846 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.035+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559c7893cb0 con 0x5559c7950220 2024-02-26T06:29:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.037+0000 7f4677400640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c7950220 0x5559c7952640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.038+0000 7f4677400640 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c7950220 0x5559c7952640 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918b0fe9bb59b87a server_cookie=7c143c98b8dd96bf in_seq=0 out_seq=0 2024-02-26T06:29:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.038+0000 7f4661600640 10 client.25846 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.042+0000 7f4661600640 1 -- 172.21.15.6:0/321728625 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559c7893cb0 con 0x5559c7950220 2024-02-26T06:29:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.042+0000 7f4661600640 10 client.25846 handle_command_reply: tid=0 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.042+0000 7f4681c1c740 1 client.25846 shutdown 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 msgr2=0x5559c793d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 0x5559c793d2b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4668000c00 tx=0x7f466802c670 comp rx=0 tx=0).stop 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 shutdown_connections 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c7950220 0x5559c7952640 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c793dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4661600640 0 client.25846 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 --2- 172.21.15.6:0/321728625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7936d80 0x5559c793d2b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 >> 172.21.15.6:0/321728625 conn(0x5559c78fe480 msgr2=0x5559c79337c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 shutdown_connections 2024-02-26T06:29:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 1 -- 172.21.15.6:0/321728625 wait complete. 2024-02-26T06:29:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.043+0000 7f4681c1c740 20 client.25846 trim_cache size 0 max 16384 2024-02-26T06:29:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.060+0000 7f4681c1c740 1 Processor -- start 2024-02-26T06:29:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 -- start start 2024-02-26T06:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7936d80 0x5559c793f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c793a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c75e5e60 con 0x5559c7936d80 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c786fcd0 con 0x5559c7937af0 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4681c1c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c7923670 con 0x5559c7940300 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4677e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7936d80 0x5559c793f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4677400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4677400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60904/0 (socket says 172.21.15.6:60904) 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.061+0000 7f4677400640 1 -- 172.21.15.6:0/3309406171 learned_addr learned my addr 172.21.15.6:0/3309406171 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f467cc00640 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c793a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677400640 1 -- 172.21.15.6:0/3309406171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 msgr2=0x5559c793a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677400640 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c793a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677400640 1 -- 172.21.15.6:0/3309406171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7936d80 msgr2=0x5559c793f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677400640 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7936d80 0x5559c793f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677400640 1 -- 172.21.15.6:0/3309406171 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c7838500 con 0x5559c7937af0 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f467cc00640 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c793a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677e00640 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7936d80 0x5559c793f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4677400640 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793db50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4664097260 tx=0x7f466409b500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a3e75048bf6ab9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4663e00640 1 -- 172.21.15.6:0/3309406171 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466409d050 con 0x5559c7937af0 2024-02-26T06:29:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4663e00640 1 -- 172.21.15.6:0/3309406171 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f466409abd0 con 0x5559c7937af0 2024-02-26T06:29:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.062+0000 7f4663e00640 1 -- 172.21.15.6:0/3309406171 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664098f30 con 0x5559c7937af0 2024-02-26T06:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 -- 172.21.15.6:0/3309406171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 msgr2=0x5559c793db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793db50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4664097260 tx=0x7f466409b500 comp rx=0 tx=0).stop 2024-02-26T06:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 -- 172.21.15.6:0/3309406171 shutdown_connections 2024-02-26T06:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7940300 0x5559c793a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7937af0 0x5559c793db50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 --2- 172.21.15.6:0/3309406171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7936d80 0x5559c793f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 -- 172.21.15.6:0/3309406171 >> 172.21.15.6:0/3309406171 conn(0x5559c78fe480 msgr2=0x5559c793bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 -- 172.21.15.6:0/3309406171 shutdown_connections 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.063+0000 7f4681c1c740 1 -- 172.21.15.6:0/3309406171 wait complete. 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 -- start start 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 Processor -- start 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7947550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 0x5559c79458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c7945f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c79465c0 con 0x5559c7940300 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c7946730 con 0x5559c7936d80 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.064+0000 7f4681c1c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c79a6750 con 0x5559c7937af0 2024-02-26T06:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4681c1c740 10 client.0 _ll_register_callbacks cb 0x7f4668009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f467cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c7945f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 0x5559c79458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f467cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c7945f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45980/0 (socket says 172.21.15.6:45980) 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 0x5559c79458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39772/0 (socket says 172.21.15.6:39772) 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f467cc00640 1 -- 172.21.15.6:0/1816492563 learned_addr learned my addr 172.21.15.6:0/1816492563 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677e00640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7947550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 -- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 msgr2=0x5559c7947550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7947550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 -- 172.21.15.6:0/1816492563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 msgr2=0x5559c7945f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c7945f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f4677400640 1 -- 172.21.15.6:0/1816492563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559c789bc20 con 0x5559c7937af0 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.065+0000 7f467cc00640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c7945f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4677400640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 0x5559c79458f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4664004ad0 tx=0x7f4664005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c5134019d51db5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4661600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4661600640 1 -- 172.21.15.6:0/1816492563 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466409d040 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4661600640 1 -- 172.21.15.6:0/1816492563 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f46640a35a0 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559c78c2d00 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4661600640 1 -- 172.21.15.6:0/1816492563 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f466409ef50 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4681c1c740 10 client.25849 fetch_fsmap learned FSMap version 28 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5559c781ead0 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4661600640 1 -- 172.21.15.6:0/1816492563 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5559c78c2d00 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.066+0000 7f4661600640 1 -- 172.21.15.6:0/1816492563 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f466401f060 con 0x5559c7937af0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4681c1c740 10 client.25849 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4681c1c740 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4681c1c740 20 client.25849 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c79a7ab0 0x5559c79a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4681c1c740 4 client.25849 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559c759d600 con 0x5559c79a7ab0 2024-02-26T06:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4677e00640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c79a7ab0 0x5559c79a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4677e00640 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c79a7ab0 0x5559c79a9ed0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2902a9a03b3576 server_cookie=46c993be9ce07fc6 in_seq=0 out_seq=0 2024-02-26T06:29:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.067+0000 7f4661600640 10 client.25849 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4661600640 1 -- 172.21.15.6:0/1816492563 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559c759d600 con 0x5559c79a7ab0 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4661600640 10 client.25849 handle_command_reply: tid=0 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 client.25849 shutdown 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 msgr2=0x5559c79458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 0x5559c79458f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4664004ad0 tx=0x7f4664005ab0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 shutdown_connections 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5559c79a7ab0 0x5559c79a9ed0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5559c7940300 0x5559c7945f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559c7937af0 0x5559c79458f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4661600640 0 client.25849 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 --2- 172.21.15.6:0/1816492563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5559c7936d80 0x5559c7947550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 >> 172.21.15.6:0/1816492563 conn(0x5559c78fe480 msgr2=0x5559c7930480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 shutdown_connections 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 1 -- 172.21.15.6:0/1816492563 wait complete. 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.136+0000 7f4681c1c740 20 client.25849 trim_cache size 0 max 16384 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:29:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f465c0a22b0 msgr2=0x7f465c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f465c0a22b0 0x7f465c0a4770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4670010610 tx=0x7f4670004a40 comp rx=0 tx=0).stop 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 msgr2=0x7f467819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 0x7f467819fbf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f466400e9f0 tx=0x7f466400eeb0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 shutdown_connections 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f465c0a22b0 0x7f465c0a4770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46781998b0 0x7f4678199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46781086a0 0x7f467819fbf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 --2- 172.21.15.6:0/1132042059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46781035d0 0x7f467819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 >> 172.21.15.6:0/1132042059 conn(0x7f46780fee70 msgr2=0x7f46781011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.137+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 shutdown_connections 2024-02-26T06:29:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:02.138+0000 7f467ec00640 1 -- 172.21.15.6:0/1132042059 wait complete. 2024-02-26T06:29:02.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:01 smithi167 ceph-mon[36229]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-26T06:29:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:01 smithi090 ceph-mon[35681]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-26T06:29:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:01 smithi006 ceph-mon[31790]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-26T06:29:02.943 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T06:29:02.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:29:02.943 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 -- 172.21.15.6:0/468867974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748075fd0 msgr2=0x7f7748111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 --2- 172.21.15.6:0/468867974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748075fd0 0x7f7748111a80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7730009900 tx=0x7f773002bc40 comp rx=0 tx=0).stop 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 -- 172.21.15.6:0/468867974 shutdown_connections 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 --2- 172.21.15.6:0/468867974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7748111fc0 0x7f77481143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 --2- 172.21.15.6:0/468867974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748075fd0 0x7f7748111a80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 --2- 172.21.15.6:0/468867974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77480756b0 0x7f7748075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 -- 172.21.15.6:0/468867974 >> 172.21.15.6:0/468867974 conn(0x7f77480fedd0 msgr2=0x7f77481011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 -- 172.21.15.6:0/468867974 shutdown_connections 2024-02-26T06:29:03.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.172+0000 7f774f200640 1 -- 172.21.15.6:0/468867974 wait complete. 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.173+0000 7f774f200640 1 Processor -- start 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 -- start start 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77480756b0 0x7f774819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 0x7f774819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77481a2890 0x7f77481a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77481a3280 con 0x7f7748111fc0 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77481a33f0 con 0x7f77481a2890 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77481a3560 con 0x7f77480756b0 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f7747400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 0x7f774819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f7747400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 0x7f774819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45998/0 (socket says 172.21.15.6:45998) 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f7747400640 1 -- 172.21.15.6:0/329609356 learned_addr learned my addr 172.21.15.6:0/329609356 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f7747e00640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77480756b0 0x7f774819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.174+0000 7f774d200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77481a2890 0x7f77481a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7747400640 1 -- 172.21.15.6:0/329609356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77480756b0 msgr2=0x7f774819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7747400640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77480756b0 0x7f774819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7747400640 1 -- 172.21.15.6:0/329609356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77481a2890 msgr2=0x7f77481a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7747400640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77481a2890 0x7f77481a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7747400640 1 -- 172.21.15.6:0/329609356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738009840 con 0x7f7748111fc0 2024-02-26T06:29:03.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7747400640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 0x7f774819e510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7730005ab0 tx=0x7f773002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.175+0000 7f7744c00640 1 -- 172.21.15.6:0/329609356 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730004350 con 0x7f7748111fc0 2024-02-26T06:29:03.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.176+0000 7f7744c00640 1 -- 172.21.15.6:0/329609356 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77300044b0 con 0x7f7748111fc0 2024-02-26T06:29:03.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.176+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730009560 con 0x7f7748111fc0 2024-02-26T06:29:03.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.176+0000 7f7744c00640 1 -- 172.21.15.6:0/329609356 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730003c50 con 0x7f7748111fc0 2024-02-26T06:29:03.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.176+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77481aade0 con 0x7f7748111fc0 2024-02-26T06:29:03.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.179+0000 7f7744c00640 1 -- 172.21.15.6:0/329609356 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f77300054f0 con 0x7f7748111fc0 2024-02-26T06:29:03.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.179+0000 7f77521ac740 1 Processor -- start 2024-02-26T06:29:03.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.179+0000 7f77521ac740 1 -- start start 2024-02-26T06:29:03.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f77521ac740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde41f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f77521ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde427ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f77521ac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f77521ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde3da2d0 con 0x55adde41f830 2024-02-26T06:29:03.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f77521ac740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde38ce20 con 0x55adde41ef10 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f77521ac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde38a350 con 0x55adde428410 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f7747e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde427ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f7747e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde427ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46002/0 (socket says 172.21.15.6:46002) 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f7747e00640 1 -- 172.21.15.6:0/507445820 learned_addr learned my addr 172.21.15.6:0/507445820 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f774d200640 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde41f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.180+0000 7f7747400640 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7747e00640 1 -- 172.21.15.6:0/507445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 msgr2=0x55adde42a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7747e00640 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7747e00640 1 -- 172.21.15.6:0/507445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 msgr2=0x55adde41f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7747e00640 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde41f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7747e00640 1 -- 172.21.15.6:0/507445820 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adde34ec50 con 0x55adde41f830 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f774d200640 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde41f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7744c00640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f772c0a2350 0x7f772c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7747e00640 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde427ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7738011680 tx=0x7f7738011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df90b780adbf841a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f7744c00640 1 -- 172.21.15.6:0/329609356 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f77300f06b0 con 0x7f7748111fc0 2024-02-26T06:29:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f774d200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f772c0a2350 0x7f772c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.181+0000 7f773d600640 1 -- 172.21.15.6:0/507445820 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738003c30 con 0x55adde41f830 2024-02-26T06:29:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.182+0000 7f773d600640 1 -- 172.21.15.6:0/507445820 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f773800fe20 con 0x55adde41f830 2024-02-26T06:29:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.182+0000 7f773d600640 1 -- 172.21.15.6:0/507445820 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77380193e0 con 0x55adde41f830 2024-02-26T06:29:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.182+0000 7f77521ac740 1 -- 172.21.15.6:0/507445820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 msgr2=0x55adde427ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.182+0000 7f774d200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f772c0a2350 0x7f772c0a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f774806a4d0 tx=0x7f7740005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.182+0000 7f77521ac740 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde427ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7738011680 tx=0x7f7738011b40 comp rx=0 tx=0).stop 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 -- 172.21.15.6:0/507445820 shutdown_connections 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde427ed0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 --2- 172.21.15.6:0/507445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde41f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 -- 172.21.15.6:0/507445820 >> 172.21.15.6:0/507445820 conn(0x55adde419640 msgr2=0x55adde419a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 -- 172.21.15.6:0/507445820 shutdown_connections 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 -- 172.21.15.6:0/507445820 wait complete. 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 -- start start 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.183+0000 7f77521ac740 1 Processor -- start 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde4253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde425920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde425e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde385b30 con 0x55adde41ef10 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde32afa0 con 0x55adde428410 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde32ac40 con 0x55adde41f830 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f77521ac740 10 client.0 _ll_register_callbacks cb 0x55adde096c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f7747e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde4253e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f774d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde425e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f7747e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde4253e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46016/0 (socket says 172.21.15.6:46016) 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f7747e00640 1 -- 172.21.15.6:0/3258108373 learned_addr learned my addr 172.21.15.6:0/3258108373 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.184+0000 7f7747400640 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde425920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f774d200640 1 -- 172.21.15.6:0/3258108373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 msgr2=0x55adde425920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f774d200640 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde425920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f774d200640 1 -- 172.21.15.6:0/3258108373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 msgr2=0x55adde4253e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f774d200640 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde4253e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f774d200640 1 -- 172.21.15.6:0/3258108373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7738009e10 con 0x55adde428410 2024-02-26T06:29:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f774d200640 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde425e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7740004610 tx=0x7f7740004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154351de185a9ba2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f7736000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f7736000640 1 -- 172.21.15.6:0/3258108373 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740017070 con 0x55adde428410 2024-02-26T06:29:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adde085860 con 0x55adde428410 2024-02-26T06:29:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.185+0000 7f7736000640 1 -- 172.21.15.6:0/3258108373 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7740005020 con 0x55adde428410 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f7736000640 1 -- 172.21.15.6:0/3258108373 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77400051e0 con 0x55adde428410 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 10 client.25883 fetch_fsmap learned FSMap version 28 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55adde3841b0 con 0x55adde428410 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f7736000640 1 -- 172.21.15.6:0/3258108373 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55adde085860 con 0x55adde428410 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f7736000640 1 -- 172.21.15.6:0/3258108373 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f774001f640 con 0x55adde428410 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 10 client.25883 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 20 client.25883 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde438900 0x55adde43ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 4 client.25883 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.186+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adde0d6410 con 0x55adde438900 2024-02-26T06:29:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.187+0000 7f7747e00640 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde438900 0x55adde43ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.187+0000 7f7747e00640 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde438900 0x55adde43ad20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6f1db7aa56deb4 server_cookie=b43bd6bfb8331991 in_seq=0 out_seq=0 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.187+0000 7f7736000640 10 client.25883 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f7736000640 1 -- 172.21.15.6:0/3258108373 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adde0d6410 con 0x55adde438900 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f7736000640 10 client.25883 handle_command_reply: tid=0 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 client.25883 shutdown 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 msgr2=0x55adde425e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde425e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7740004610 tx=0x7f7740004730 comp rx=0 tx=0).stop 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 shutdown_connections 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde438900 0x55adde43ad20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde425e60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde425920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 --2- 172.21.15.6:0/3258108373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde4253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f7736000640 0 client.25883 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 >> 172.21.15.6:0/3258108373 conn(0x55adde419640 msgr2=0x55adde41b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 shutdown_connections 2024-02-26T06:29:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 1 -- 172.21.15.6:0/3258108373 wait complete. 2024-02-26T06:29:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.188+0000 7f77521ac740 20 client.25883 trim_cache size 0 max 16384 2024-02-26T06:29:03.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 Processor -- start 2024-02-26T06:29:03.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 -- start start 2024-02-26T06:29:03.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde425c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde41b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde42f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde0ce0c0 con 0x55adde41ef10 2024-02-26T06:29:03.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde41e330 con 0x55adde428410 2024-02-26T06:29:03.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.205+0000 7f77521ac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde418e40 con 0x55adde41f830 2024-02-26T06:29:03.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde425c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f774d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde42f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde425c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46028/0 (socket says 172.21.15.6:46028) 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 -- 172.21.15.6:0/4041903355 learned_addr learned my addr 172.21.15.6:0/4041903355 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747400640 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde41b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 -- 172.21.15.6:0/4041903355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 msgr2=0x55adde41b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde41b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 -- 172.21.15.6:0/4041903355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 msgr2=0x55adde42f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.206+0000 7f7747e00640 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde42f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f7747e00640 1 -- 172.21.15.6:0/4041903355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adde320a60 con 0x55adde41ef10 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f774d200640 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde42f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f7747400640 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde41b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f7747e00640 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde425c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7738004520 tx=0x7f7738004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb17f3ab76a092e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f773cc00640 1 -- 172.21.15.6:0/4041903355 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800f8c0 con 0x55adde41ef10 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f773cc00640 1 -- 172.21.15.6:0/4041903355 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f773800fa20 con 0x55adde41ef10 2024-02-26T06:29:03.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.207+0000 7f773cc00640 1 -- 172.21.15.6:0/4041903355 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738004930 con 0x55adde41ef10 2024-02-26T06:29:03.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 -- 172.21.15.6:0/4041903355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 msgr2=0x55adde425c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde425c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7738004520 tx=0x7f7738004550 comp rx=0 tx=0).stop 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 -- 172.21.15.6:0/4041903355 shutdown_connections 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde428410 0x55adde42f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde41f830 0x55adde41b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 --2- 172.21.15.6:0/4041903355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41ef10 0x55adde425c50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 -- 172.21.15.6:0/4041903355 >> 172.21.15.6:0/4041903355 conn(0x55adde419640 msgr2=0x55adde41aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 -- 172.21.15.6:0/4041903355 shutdown_connections 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 -- 172.21.15.6:0/4041903355 wait complete. 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.208+0000 7f77521ac740 1 -- start start 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 Processor -- start 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde4881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde4886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde42ebb0 con 0x55adde41f830 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde42ed20 con 0x55adde41ef10 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adde42eee0 con 0x55adde428410 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f77521ac740 10 client.0 _ll_register_callbacks cb 0x7f7738009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f7747e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde4881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f7747400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde4886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f7747e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde4881a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60972/0 (socket says 172.21.15.6:60972) 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f7747e00640 1 -- 172.21.15.6:0/1222229162 learned_addr learned my addr 172.21.15.6:0/1222229162 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.209+0000 7f774d200640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747e00640 1 -- 172.21.15.6:0/1222229162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 msgr2=0x55adde42e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747e00640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747e00640 1 -- 172.21.15.6:0/1222229162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 msgr2=0x55adde4886e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747e00640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde4886e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747e00640 1 -- 172.21.15.6:0/1222229162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77481aade0 con 0x55adde41ef10 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747400640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde4886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7747e00640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde4881a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7738011b80 tx=0x7f7738009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b16fe0c38a7498f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7736000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7736000640 1 -- 172.21.15.6:0/1222229162 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738003970 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f7736000640 1 -- 172.21.15.6:0/1222229162 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7738003ad0 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.210+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f773800fe20 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f7736000640 1 -- 172.21.15.6:0/1222229162 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7738003c90 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 10 client.25889 fetch_fsmap learned FSMap version 28 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55adde3841b0 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f7736000640 1 -- 172.21.15.6:0/1222229162 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f773800fe20 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f7736000640 1 -- 172.21.15.6:0/1222229162 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7738005040 con 0x55adde41ef10 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 10 client.25889 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 20 client.25889 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde490d30 0x55adde493150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 4 client.25889 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.211+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55adde0c27b0 con 0x55adde490d30 2024-02-26T06:29:03.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.212+0000 7f7747400640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde490d30 0x55adde493150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.212+0000 7f7747400640 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde490d30 0x55adde493150 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861ec14981ced0dd server_cookie=89503e22c0fff5dd in_seq=0 out_seq=0 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.212+0000 7f7736000640 10 client.25889 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f7736000640 1 -- 172.21.15.6:0/1222229162 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55adde0c27b0 con 0x55adde490d30 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f7736000640 10 client.25889 handle_command_reply: tid=0 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 client.25889 shutdown 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 msgr2=0x55adde4881a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde4881a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7738011b80 tx=0x7f7738009590 comp rx=0 tx=0).stop 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 shutdown_connections 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55adde490d30 0x55adde493150 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adde428410 0x55adde42e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adde41f830 0x55adde4886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 --2- 172.21.15.6:0/1222229162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55adde41ef10 0x55adde4881a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f7736000640 0 client.25889 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 >> 172.21.15.6:0/1222229162 conn(0x55adde419640 msgr2=0x55adde41b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 shutdown_connections 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 1 -- 172.21.15.6:0/1222229162 wait complete. 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.213+0000 7f77521ac740 20 client.25889 trim_cache size 0 max 16384 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "b60c94fa-a882-49ce-8456-0a8868b0d513", 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2024-02-26T06:29:03.217 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f772c0a2350 msgr2=0x7f772c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f772c0a2350 0x7f772c0a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f774806a4d0 tx=0x7f7740005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 msgr2=0x7f774819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 0x7f774819e510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7730005ab0 tx=0x7f773002c6e0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 shutdown_connections 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f772c0a2350 0x7f772c0a4810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77481a2890 0x7f77481a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7748111fc0 0x7f774819e510 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 --2- 172.21.15.6:0/329609356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77480756b0 0x7f774819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 >> 172.21.15.6:0/329609356 conn(0x7f77480fedd0 msgr2=0x7f77481136b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 shutdown_connections 2024-02-26T06:29:03.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.214+0000 7f774f200640 1 -- 172.21.15.6:0/329609356 wait complete. 2024-02-26T06:29:03.230 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:02 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:03.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:29:03.238 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:29:03.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:02 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:03.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:02 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:03.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.449+0000 7f4a8c000640 1 -- 172.21.15.6:0/4117722154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 msgr2=0x7f4a84103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.449+0000 7f4a8c000640 1 --2- 172.21.15.6:0/4117722154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a84103970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009890 tx=0x7f4a7802bb70 comp rx=0 tx=0).stop 2024-02-26T06:29:03.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.449+0000 7f4a8c000640 1 -- 172.21.15.6:0/4117722154 shutdown_connections 2024-02-26T06:29:03.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.450+0000 7f4a8c000640 1 --2- 172.21.15.6:0/4117722154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 0x7f4a84110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.450+0000 7f4a8c000640 1 --2- 172.21.15.6:0/4117722154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a84103f40 0x7f4a84107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.450+0000 7f4a8c000640 1 --2- 172.21.15.6:0/4117722154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a84103970 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.450+0000 7f4a8c000640 1 -- 172.21.15.6:0/4117722154 >> 172.21.15.6:0/4117722154 conn(0x7f4a840fedd0 msgr2=0x7f4a841011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.450+0000 7f4a8c000640 1 -- 172.21.15.6:0/4117722154 shutdown_connections 2024-02-26T06:29:03.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.450+0000 7f4a8c000640 1 -- 172.21.15.6:0/4117722154 wait complete. 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 Processor -- start 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 -- start start 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a8419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 0x7f4a8419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a841997e0 0x7f4a84199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8419a290 con 0x7f4a841997e0 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8419a400 con 0x7f4a84103590 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.451+0000 7f4a8c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8419a570 con 0x7f4a84108660 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a8419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a841997e0 0x7f4a84199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a8419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:60984/0 (socket says 172.21.15.6:60984) 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 -- 172.21.15.6:0/1196206842 learned_addr learned my addr 172.21.15.6:0/1196206842 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a88c00640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 0x7f4a8419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 -- 172.21.15.6:0/1196206842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 msgr2=0x7f4a8419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 0x7f4a8419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 -- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a841997e0 msgr2=0x7f4a84199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a841997e0 0x7f4a84199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 -- 172.21.15.6:0/1196206842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70009560 con 0x7f4a84103590 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a88c00640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 0x7f4a8419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a8a000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a841997e0 0x7f4a84199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.452+0000 7f4a89600640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a8419f670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a78037040 tx=0x7f4a780045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.453+0000 7f4a82000640 1 -- 172.21.15.6:0/1196206842 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780349d0 con 0x7f4a84103590 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.453+0000 7f4a82000640 1 -- 172.21.15.6:0/1196206842 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a7802c9f0 con 0x7f4a84103590 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.453+0000 7f4a82000640 1 -- 172.21.15.6:0/1196206842 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780035f0 con 0x7f4a84103590 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.453+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a78009490 con 0x7f4a84103590 2024-02-26T06:29:03.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.453+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a841a66a0 con 0x7f4a84103590 2024-02-26T06:29:03.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.457+0000 7f4a8ee26740 1 Processor -- start 2024-02-26T06:29:03.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.457+0000 7f4a8ee26740 1 -- start start 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.457+0000 7f4a82000640 1 -- 172.21.15.6:0/1196206842 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f4a7803b040 con 0x7f4a84103590 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b25394cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b25395710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25304120 con 0x562b2539e600 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25301650 con 0x562b25395f70 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b252fce30 con 0x562b25395290 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46062/0 (socket says 172.21.15.6:46062) 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b25394cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 -- 172.21.15.6:0/3956017948 learned_addr learned my addr 172.21.15.6:0/3956017948 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a89600640 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b25395710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 -- 172.21.15.6:0/3956017948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 msgr2=0x562b25395710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b25395710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 -- 172.21.15.6:0/3956017948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 msgr2=0x562b25394cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b25394cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.458+0000 7f4a88c00640 1 -- 172.21.15.6:0/3956017948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b252b6030 con 0x562b2539e600 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a8a000640 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b25394cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a88c00640 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a09f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009df0 tx=0x7f4a700117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1110844de27a7bb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a82000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4a6c0a23d0 0x7f4a6c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a76600640 1 -- 172.21.15.6:0/3956017948 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70012650 con 0x562b2539e600 2024-02-26T06:29:03.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a76600640 1 -- 172.21.15.6:0/3956017948 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a70011ec0 con 0x562b2539e600 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a82000640 1 -- 172.21.15.6:0/1196206842 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4a780eff10 con 0x7f4a84103590 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a76600640 1 -- 172.21.15.6:0/3956017948 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000fd00 con 0x562b2539e600 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.459+0000 7f4a8a000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4a6c0a23d0 0x7f4a6c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 -- 172.21.15.6:0/3956017948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 msgr2=0x562b253a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a09f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009df0 tx=0x7f4a700117b0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 -- 172.21.15.6:0/3956017948 shutdown_connections 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a09f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b25395710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/3956017948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b25394cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 -- 172.21.15.6:0/3956017948 >> 172.21.15.6:0/3956017948 conn(0x562b2535c570 msgr2=0x562b2538fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 -- 172.21.15.6:0/3956017948 shutdown_connections 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8a000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4a6c0a23d0 0x7f4a6c0a4890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009fd0 tx=0x7f4a7c009500 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.460+0000 7f4a8ee26740 1 -- 172.21.15.6:0/3956017948 wait complete. 2024-02-26T06:29:03.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 -- start start 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 Processor -- start 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b2539bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b252a1e90 con 0x562b2539e600 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b252a1b30 con 0x562b25395f70 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2506b1c0 con 0x562b25395290 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.461+0000 7f4a8ee26740 10 client.0 _ll_register_callbacks cb 0x562b252b5930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b2539bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a89600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:32780/0 (socket says 172.21.15.6:32780) 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a89600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:39878/0 (socket says 172.21.15.6:39878) 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 -- 172.21.15.6:0/1966475820 learned_addr learned my addr 172.21.15.6:0/1966475820 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 -- 172.21.15.6:0/1966475820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 msgr2=0x562b2539b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 -- 172.21.15.6:0/1966475820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 msgr2=0x562b2539bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b2539bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a88c00640 1 -- 172.21.15.6:0/1966475820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a70009a20 con 0x562b25395f70 2024-02-26T06:29:03.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a8a000640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b2539bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.462+0000 7f4a89600640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a88c00640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539ba30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009dc0 tx=0x7f4a70005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d925bd2f94bb6d75 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a6aa00640 1 -- 172.21.15.6:0/1966475820 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000fec0 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b25039720 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a6aa00640 1 -- 172.21.15.6:0/1966475820 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4a7000f440 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a6aa00640 1 -- 172.21.15.6:0/1966475820 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a700046c0 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a8ee26740 10 client.25901 fetch_fsmap learned FSMap version 28 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.463+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562b252fb4b0 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a6aa00640 1 -- 172.21.15.6:0/1966475820 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562b25039720 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a6aa00640 1 -- 172.21.15.6:0/1966475820 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4a70022040 con 0x562b25395f70 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a8ee26740 10 client.25901 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a8ee26740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a8ee26740 20 client.25901 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b253ae580 0x562b253b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a8ee26740 4 client.25901 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:03.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.464+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b25088890 con 0x562b253ae580 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.465+0000 7f4a89600640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b253ae580 0x562b253b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.465+0000 7f4a89600640 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b253ae580 0x562b253b09c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2fb9dd5beb85e6 server_cookie=63b16ef59d1361f8 in_seq=0 out_seq=0 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.465+0000 7f4a6aa00640 10 client.25901 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a6aa00640 1 -- 172.21.15.6:0/1966475820 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b25088890 con 0x562b253ae580 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a6aa00640 10 client.25901 handle_command_reply: tid=0 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 client.25901 shutdown 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 msgr2=0x562b2539ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539ba30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009dc0 tx=0x7f4a70005270 comp rx=0 tx=0).stop 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 shutdown_connections 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b253ae580 0x562b253b09c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b2539bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539ba30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/1966475820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a6aa00640 0 client.25901 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 >> 172.21.15.6:0/1966475820 conn(0x562b2535c570 msgr2=0x562b25391a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 shutdown_connections 2024-02-26T06:29:03.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.466+0000 7f4a8ee26740 1 -- 172.21.15.6:0/1966475820 wait complete. 2024-02-26T06:29:03.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.467+0000 7f4a8ee26740 20 client.25901 trim_cache size 0 max 16384 2024-02-26T06:29:03.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 Processor -- start 2024-02-26T06:29:03.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 -- start start 2024-02-26T06:29:03.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25039720 con 0x562b2539e600 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25390ae0 con 0x562b25395f70 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.483+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25394b00 con 0x562b25395290 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46086/0 (socket says 172.21.15.6:46086) 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 -- 172.21.15.6:0/4281428867 learned_addr learned my addr 172.21.15.6:0/4281428867 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a89600640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a88c00640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 -- 172.21.15.6:0/4281428867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 msgr2=0x562b2539dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 -- 172.21.15.6:0/4281428867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 msgr2=0x562b2539bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 -- 172.21.15.6:0/4281428867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b25297920 con 0x562b2539e600 2024-02-26T06:29:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a88c00640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a89600640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:03.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.484+0000 7f4a8a000640 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a5560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0123d0 tx=0x7f4a7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1315b4d09fcd4aff server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.485+0000 7f4a75c00640 1 -- 172.21.15.6:0/4281428867 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c026070 con 0x562b2539e600 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.485+0000 7f4a75c00640 1 -- 172.21.15.6:0/4281428867 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c010e10 con 0x562b2539e600 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.485+0000 7f4a75c00640 1 -- 172.21.15.6:0/4281428867 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c017070 con 0x562b2539e600 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.485+0000 7f4a8ee26740 1 -- 172.21.15.6:0/4281428867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 msgr2=0x562b253a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.485+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a5560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0123d0 tx=0x7f4a7c017000 comp rx=0 tx=0).stop 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 -- 172.21.15.6:0/4281428867 shutdown_connections 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b253a5560 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395f70 0x562b2539bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/4281428867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395290 0x562b2539dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 -- 172.21.15.6:0/4281428867 >> 172.21.15.6:0/4281428867 conn(0x562b2535c570 msgr2=0x562b2538f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 -- 172.21.15.6:0/4281428867 shutdown_connections 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 -- 172.21.15.6:0/4281428867 wait complete. 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 -- start start 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.486+0000 7f4a8ee26740 1 Processor -- start 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395290 0x562b253a5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395f70 0x562b25397f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b25398550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25398c20 con 0x562b2539e600 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25398d90 con 0x562b25395290 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25404ae0 con 0x562b25395f70 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8ee26740 10 client.0 _ll_register_callbacks cb 0x7f4a7c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b25398550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a89600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395290 0x562b253a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b25398550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46092/0 (socket says 172.21.15.6:46092) 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a8a000640 1 -- 172.21.15.6:0/891132510 learned_addr learned my addr 172.21.15.6:0/891132510 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:03.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a88c00640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395f70 0x562b25397f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a89600640 1 -- 172.21.15.6:0/891132510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395f70 msgr2=0x562b25397f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a89600640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395f70 0x562b25397f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a89600640 1 -- 172.21.15.6:0/891132510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 msgr2=0x562b25398550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a89600640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b25398550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.487+0000 7f4a89600640 1 -- 172.21.15.6:0/891132510 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c012050 con 0x562b25395290 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a8a000640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b25398550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a89600640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395290 0x562b253a5990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4a780ed570 tx=0x7f4a780f3960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=835162e51b241770 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a6aa00640 1 -- 172.21.15.6:0/891132510 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b89e0 con 0x562b25395290 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a6aa00640 1 -- 172.21.15.6:0/891132510 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4a780bce40 con 0x562b25395290 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b24ffc7e0 con 0x562b25395290 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a6aa00640 1 -- 172.21.15.6:0/891132510 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a780bc510 con 0x562b25395290 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a8ee26740 10 client.25904 fetch_fsmap learned FSMap version 28 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.488+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562b252bd400 con 0x562b25395290 2024-02-26T06:29:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a6aa00640 1 -- 172.21.15.6:0/891132510 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562b24ffc7e0 con 0x562b25395290 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a6aa00640 1 -- 172.21.15.6:0/891132510 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4a780b8040 con 0x562b25395290 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a8ee26740 10 client.25904 fetch_fsmap finished waiting for FSMap version 28 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a8ee26740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a8ee26740 20 client.25904 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b25405ee0 0x562b25408300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a8ee26740 4 client.25904 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:29:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b24fa8700 con 0x562b25405ee0 2024-02-26T06:29:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.489+0000 7f4a88c00640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b25405ee0 0x562b25408300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a88c00640 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b25405ee0 0x562b25408300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a561684f268b4aaa server_cookie=9691d1ba247b33c1 in_seq=0 out_seq=0 2024-02-26T06:29:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a6aa00640 10 client.25904 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a6aa00640 1 -- 172.21.15.6:0/891132510 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562b24fa8700 con 0x562b25405ee0 2024-02-26T06:29:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a6aa00640 10 client.25904 handle_command_reply: tid=0 2024-02-26T06:29:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 client.25904 shutdown 2024-02-26T06:29:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395290 msgr2=0x562b253a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395290 0x562b253a5990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4a780ed570 tx=0x7f4a780f3960 comp rx=0 tx=0).stop 2024-02-26T06:29:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 shutdown_connections 2024-02-26T06:29:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x562b25405ee0 0x562b25408300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b2539e600 0x562b25398550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b25395f70 0x562b25397f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a6aa00640 0 client.25904 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 --2- 172.21.15.6:0/891132510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b25395290 0x562b253a5990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.490+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 >> 172.21.15.6:0/891132510 conn(0x562b2535c570 msgr2=0x562b25399d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 shutdown_connections 2024-02-26T06:29:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8ee26740 1 -- 172.21.15.6:0/891132510 wait complete. 2024-02-26T06:29:03.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8ee26740 20 client.25904 trim_cache size 0 max 16384 2024-02-26T06:29:03.496 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:29:03.496 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (1 inodes in the stack)", 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: "b60c94fa-a882-49ce-8456-0a8868b0d513": { 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "b60c94fa-a882-49ce-8456-0a8868b0d513", 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:29:03.497 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4a6c0a23d0 msgr2=0x7f4a6c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4a6c0a23d0 0x7f4a6c0a4890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009fd0 tx=0x7f4a7c009500 comp rx=0 tx=0).stop 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 msgr2=0x7f4a8419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a8419f670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a78037040 tx=0x7f4a780045b0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 shutdown_connections 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4a6c0a23d0 0x7f4a6c0a4890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a841997e0 0x7f4a84199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84108660 0x7f4a8419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 --2- 172.21.15.6:0/1196206842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a84103590 0x7f4a8419f670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.491+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 >> 172.21.15.6:0/1196206842 conn(0x7f4a840fedd0 msgr2=0x7f4a84101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:03.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.492+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 shutdown_connections 2024-02-26T06:29:03.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:03.492+0000 7f4a8c000640 1 -- 172.21.15.6:0/1196206842 wait complete. 2024-02-26T06:29:03.511 INFO:tasks.cephfs.filesystem:scrub status for tag:b60c94fa-a882-49ce-8456-0a8868b0d513 - {'path': '/', 'tag': 'b60c94fa-a882-49ce-8456-0a8868b0d513', 'options': 'recursive,force'} 2024-02-26T06:29:04.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:03 smithi167 ceph-mon[36229]: pgmap v31: 97 pgs: 97 active+clean; 9.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 729 KiB/s wr, 22 op/s 2024-02-26T06:29:04.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:03 smithi167 ceph-mon[36229]: mgrmap e27: x(active, since 56s), standbys: z, y 2024-02-26T06:29:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:03 smithi090 ceph-mon[35681]: pgmap v31: 97 pgs: 97 active+clean; 9.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 729 KiB/s wr, 22 op/s 2024-02-26T06:29:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:03 smithi090 ceph-mon[35681]: mgrmap e27: x(active, since 56s), standbys: z, y 2024-02-26T06:29:04.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:03 smithi006 ceph-mon[31790]: pgmap v31: 97 pgs: 97 active+clean; 9.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 729 KiB/s wr, 22 op/s 2024-02-26T06:29:04.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:03 smithi006 ceph-mon[31790]: mgrmap e27: x(active, since 56s), standbys: z, y 2024-02-26T06:29:05.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:04 smithi167 ceph-mon[36229]: scrub queued for path: / 2024-02-26T06:29:05.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:04 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:05.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:04 smithi167 ceph-mon[36229]: scrub summary: active paths [/] 2024-02-26T06:29:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:04 smithi090 ceph-mon[35681]: scrub queued for path: / 2024-02-26T06:29:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:04 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:05.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:04 smithi090 ceph-mon[35681]: scrub summary: active paths [/] 2024-02-26T06:29:05.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:04 smithi006 ceph-mon[31790]: scrub queued for path: / 2024-02-26T06:29:05.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:04 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:05.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:04 smithi006 ceph-mon[31790]: scrub summary: active paths [/] 2024-02-26T06:29:06.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:05 smithi167 ceph-mon[36229]: pgmap v32: 97 pgs: 97 active+clean; 16 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-02-26T06:29:06.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:05 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:06.245 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.072+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/ auth{0=1,1=1,3=1,4=1} v9 ap=1 f(v0 m2024-02-26T06:28:57.943710+0000 1=0+1) n(v1 rc2024-02-26T06:29:01.355672+0000 b8912896 395=361+34)/n(v0 rc2024-02-26T06:29:00.559681+0000 b8912896 32=20+12) old_inodes=1 (inest lock dirty) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@1},l=25277 | dirtyscattered=1 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5599d3d47180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000006:[<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000006:[<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:28:57.943710+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:01.355672+0000 b8912896 395=361+34)","memory_value.dirstat":"f(v0 m2024-02-26T06:28:57.943710+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-02-26T06:29:01.355672+0000 b8912896 395=361+34)","error_str":""},"return_code":0} 2024-02-26T06:29:06.245 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.074+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ auth{0=1,1=1,3=1,4=1} v77 pv79 ap=2 f(v0 m2024-02-26T06:29:01.355672+0000 34=21+13)->f(v0 m2024-02-26T06:29:01.355672+0000 34=21+13) n(v1 rc2024-02-26T06:29:01.522670+0000 b8912896 548=514+34)/n(v0 rc2024-02-26T06:29:01.355672+0000 b8912896 394=361+33)->n(v1 rc2024-02-26T06:29:02.125664+0000 b8912896 563=529+34) old_inodes=1 (isnap sync r=1) (inest lock w=1) (ipolicy sync r=1) (ifile excl) (iversion lock w=1 last_client=25277) caps={25277=pAsLsXsFsx/pAsLsXsFsx@1},l=25277 | lock=4 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x5599d3d47700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000007:[<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000007:[<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:01.355672+0000 34=21+13)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:01.522670+0000 b8912896 548=514+34)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:01.355672+0000 34=21+13)","memory_value.rstat":"n(v1 rc2024-02-26T06:29:01.522670+0000 b8912896 548=514+34)","error_str":""},"return_code":0} 2024-02-26T06:29:06.245 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.074+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/MAINTAINERS auth v154 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d445d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000241:[<0x10000000007/MAINTAINERS v154>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000241:[<0x10000000007/MAINTAINERS v154>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.245 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.075+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d9 [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/src/ auth{0=1,1=1} fragtree_t(*^3) v782 ap=1 f(v0 m2024-02-26T06:29:01.354672+0000 156=152+4) n(v1 rc2024-02-26T06:29:01.354672+0000 201=196+5) old_inodes=1 (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=8 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5599d414db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002d9:[<0x10000000007/src v764>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000002d9:[<0x10000000007/src v782>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:01.354672+0000 156=152+4)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:01.354672+0000 201=196+5)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:01.354672+0000 156=152+4)","memory_value.rstat":"n(v1 rc2024-02-26T06:29:01.354672+0000 201=196+5)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.076+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a2 [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/ auth{0=1} v772 pv784 ap=2 f(v0 m2024-02-26T06:29:01.357672+0000 2=1+1)->f(v0 m2024-02-26T06:29:01.357672+0000 2=1+1) n(v0 rc2024-02-26T06:29:01.522670+0000 154=152+2)->n(v0 rc2024-02-26T06:29:02.125664+0000 169=167+2) (isnap sync r=1) (inest lock w=1 dirty) (ipolicy sync r=1) (ifile excl) (iversion lock w=1 last_client=25277) caps={25277=pAsLsXsFsx/pAsLsXsFsx@1},l=25277 | dirtyscattered=1 lock=4 importingcaps=0 dirfrag=1 caps=1 replicated=1 authpin=1 scrubqueue=0 0x5599d445d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000003a2:[<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000003a2:[<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:01.357672+0000 2=1+1)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:01.522670+0000 154=152+2)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:01.357672+0000 2=1+1)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:01.522670+0000 154=152+2)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.076+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028d [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/doc/ auth{0=1} v310 ap=1 f(v0 m2024-02-26T06:29:00.935677+0000 6=6+0) n(v0 rc2024-02-26T06:29:00.935677+0000 7=6+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4184000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000028d:[<0x10000000007/doc v310>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x1000000028d:[<0x10000000007/doc v310>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.935677+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.935677+0000 7=6+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.935677+0000 6=6+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.935677+0000 7=6+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.077+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000249 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.selftest auth v170 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d445cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000249:[<0x10000000007/README.selftest v170>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000249:[<0x10000000007/README.selftest v170>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.077+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002a6 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lib/ auth{0=1} v376 ap=1 f(v0 m2024-02-26T06:29:00.987676+0000 14=14+0) n(v0 rc2024-02-26T06:29:00.987676+0000 15=14+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4184580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002a6:[<0x10000000007/lib v376>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000002a6:[<0x10000000007/lib v376>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.987676+0000 14=14+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.987676+0000 15=14+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.987676+0000 14=14+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.987676+0000 15=14+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.077+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000294 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/include/ auth{0=1} v346 ap=1 f(v0 m2024-02-26T06:29:00.963677+0000 17=17+0) n(v0 rc2024-02-26T06:29:00.963677+0000 18=17+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4184b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000294:[<0x10000000007/include v346>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000294:[<0x10000000007/include v346>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.963677+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.963677+0000 18=17+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.963677+0000 17=17+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.963677+0000 18=17+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.078+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000245 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.config-sections auth v162 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d445c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000245:[<0x10000000007/README.config-sections v162>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000245:[<0x10000000007/README.config-sections v162>,<0x10000000006/xfstests-dev v77>,<0x10000000005/tmp v9>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.080+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000024b [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/acinclude.m4 auth v174 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43da000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000024b:[<0x10000000007/acinclude.m4 v174>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000024b:[<0x10000000007/acinclude.m4 v174>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.081+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d4 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/new auth v440 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43db600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002d4:[<0x10000000007/new v440>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002d4:[<0x10000000007/new v440>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.082+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makefile auth v156 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43da580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000242:[<0x10000000007/Makefile v156>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000242:[<0x10000000007/Makefile v156>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.082+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c1 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/m4/ auth{0=1} v438 ap=1 f(v0 m2024-02-26T06:29:01.023676+0000 18=18+0) n(v0 rc2024-02-26T06:29:01.023676+0000 19=18+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4185080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002c1:[<0x10000000007/m4 v438>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000002c1:[<0x10000000007/m4 v438>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:01.023676+0000 18=18+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:01.023676+0000 19=18+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:01.023676+0000 18=18+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:01.023676+0000 19=18+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.083+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d5 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/randomize.awk auth v442 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43db080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002d5:[<0x10000000007/randomize.awk v442>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002d5:[<0x10000000007/randomize.awk v442>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.083+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000289 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/crash/ auth{0=1} v296 ap=1 f(v0 m2024-02-26T06:29:00.925677+0000 3=3+0) n(v0 rc2024-02-26T06:29:00.925677+0000 4=3+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4185600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000289:[<0x10000000007/crash v296>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000289:[<0x10000000007/crash v296>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.925677+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.925677+0000 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.925677+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.925677+0000 4=3+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.246 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.083+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002b7 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ltp/ auth{0=1} v400 ap=1 f(v0 m2024-02-26T06:29:01.002676+0000 9=9+0) n(v0 rc2024-02-26T06:29:01.002676+0000 10=9+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4185b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002b7:[<0x10000000007/ltp v400>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000002b7:[<0x10000000007/ltp v400>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:01.002676+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:01.002676+0000 10=9+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:01.002676+0000 9=9+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:01.002676+0000 10=9+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.249 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.084+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002b5 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/local.config.example auth v378 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43dab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002b5:[<0x10000000007/local.config.example v378>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002b5:[<0x10000000007/local.config.example v378>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.249 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.084+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makepkgs auth v158 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43dbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000243:[<0x10000000007/Makepkgs v158>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000243:[<0x10000000007/Makepkgs v158>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.084+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000247 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.fuse auth v166 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43dc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000247:[<0x10000000007/README.fuse v166>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000247:[<0x10000000007/README.fuse v166>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.085+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002b6 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lsqa.pl auth v380 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43cec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002b6:[<0x10000000007/lsqa.pl v380>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002b6:[<0x10000000007/lsqa.pl v380>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.086+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000244 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README auth v160 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43ce680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000244:[<0x10000000007/README v160>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000244:[<0x10000000007/README v160>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.086+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d6 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/release.sh auth v444 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43cdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002d6:[<0x10000000007/release.sh v444>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002d6:[<0x10000000007/release.sh v444>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.087+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000246 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.device-mapper auth v164 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43ce100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000246:[<0x10000000007/README.device-mapper v164>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000246:[<0x10000000007/README.device-mapper v164>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.087+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000288 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configure.ac auth v288 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43cf700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000288:[<0x10000000007/configure.ac v288>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000288:[<0x10000000007/configure.ac v288>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.087+0000 7fc6a7573700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000008 [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.git/ auth{0=1,1=1,3=1,4=1} v780 ap=2 RANDEPHEMERALPIN f(v0 m2024-02-26T06:29:00.814678+0000 12=6+6) n(v1 rc2024-02-26T06:29:00.814678+0000 b8912896 37=23+14) old_inodes=1 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25277=pAsLsXsFs/pAsLsXsFsx@2} | dirtyscattered=0 lock=3 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x5599d414c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000008:[<0x10000000007/.git v146>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000008:[<0x10000000007/.git v780>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.814678+0000 12=6+6)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.814678+0000 b8912896 37=23+14)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.814678+0000 12=6+6)","memory_value.rstat":"n(v1 rc2024-02-26T06:29:00.814678+0000 b8912896 37=23+14)","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.088+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000255 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/common/ auth{0=1} v280 ap=1 f(v0 m2024-02-26T06:29:00.911677+0000 47=47+0) n(v0 rc2024-02-26T06:29:00.911677+0000 48=47+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4186100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000255:[<0x10000000007/common v280>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000255:[<0x10000000007/common v280>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v10>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.911677+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.911677+0000 48=47+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.911677+0000 47=47+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.911677+0000 48=47+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.088+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d8 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/soak auth v448 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d43cf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002d8:[<0x10000000007/soak v448>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002d8:[<0x10000000007/soak v448>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.089+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/LICENSES/ auth{0=1} v152 ap=1 f(v0 m2024-02-26T06:29:00.826678+0000 1=1+0) n(v0 rc2024-02-26T06:29:00.826678+0000 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d4186680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000023f:[<0x10000000007/LICENSES v152>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x1000000023f:[<0x10000000007/LICENSES v152>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.826678+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.826678+0000 2=1+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.826678+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.826678+0000 2=1+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.089+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000248 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.overlay auth v168 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d3d44580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000248:[<0x10000000007/README.overlay v168>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000248:[<0x10000000007/README.overlay v168>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.090+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000024a [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/VERSION auth v172 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4528000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000024a:[<0x10000000007/VERSION v172>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000024a:[<0x10000000007/VERSION v172>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.090+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.gitignore auth v148 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4528580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023e:[<0x10000000007/.gitignore v148>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000023e:[<0x10000000007/.gitignore v148>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.250 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.090+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000285 [...2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configs/ auth{0=1} v286 ap=1 f(v0 m2024-02-26T06:29:00.916677+0000 2=2+0) n(v0 rc2024-02-26T06:29:00.916677+0000 3=2+1) (inest mix) (ifile excl) (iversion lock) caps={25277=pAsLsXsFsx/pAsLsXsFsx@2},l=25277 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 authpin=1 scrubqueue=0 0x5599d414f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000285:[<0x10000000007/configs v286>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000285:[<0x10000000007/configs v286>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.916677+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.916677+0000 3=2+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.916677+0000 2=2+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:00.916677+0000 3=2+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.090+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000254 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/check auth v184 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4528b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000254:[<0x10000000007/check v184>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000254:[<0x10000000007/check v184>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.091+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d7 [2,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/setup auth v446 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@1} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4529080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002d7:[<0x10000000007/setup v446>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000002d7:[<0x10000000007/setup v446>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.094+0000 7fc6a7573700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000024c [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/build/ auth{0=1} v781 ap=2 RANDEPHEMERALPIN f(v0 m2024-02-26T06:29:00.844678+0000 3=1+2) n(v1 rc2024-02-26T06:29:00.845678+0000 8=5+3) old_inodes=1 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25277=pAsLsXsFs/pAsLsXsFsx@2} | dirtyscattered=0 lock=3 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 randepin 0x5599d414d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000024c:[<0x10000000007/build v182>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x1000000024c:[<0x10000000007/build v781>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:00.844678+0000 3=1+2)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:00.845678+0000 8=5+3)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:00.844678+0000 3=1+2)","memory_value.rstat":"n(v1 rc2024-02-26T06:29:00.845678+0000 8=5+3)","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.127+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a3 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/Makefile auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4529600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a3:[<0x100000003a2/Makefile v2>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003a3:[<0x100000003a2/Makefile v2>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.130+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a4 [...3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/ auth{0=1} fragtree_t(*^3) v338 pv388 ap=27 RANDEPHEMERALPIN f(v0 m2024-02-26T06:29:02.125664+0000 166=166+0)->f(v0 m2024-02-26T06:29:06.127619+0000 191=191+0) n(v0 rc2024-02-26T06:29:02.125664+0000 167=166+1)->n(v0 rc2024-02-26T06:29:06.127619+0000 192=191+1) (iauth sync r=1) (isnap sync r=1) (inest lock w=1) (ipolicy sync r=1) (ifile excl w=1) (iversion lock w=25 last_client=25277) caps={25277=pAsLsXsFsx/pAsLsXsFsx@27},l=25277 | lock=6 importingcaps=0 dirfrag=8 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 randepin 0x5599d4529b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000003a4:[<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000003a4:[<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T06:29:02.125664+0000 166=166+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T06:29:02.125664+0000 167=166+1)","memory_value.dirstat":"f(v0 m2024-02-26T06:29:02.125664+0000 166=166+0)","memory_value.rstat":"n(v0 rc2024-02-26T06:29:02.125664+0000 167=166+1)","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.165+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a9 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003 auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a9:[<0x100000003a4/003 v10>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003a9:[<0x100000003a4/003 v10>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.165+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042a [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067.out auth v268 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000042a:[<0x100000003a4/067.out v268>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000042a:[<0x100000003a4/067.out v268>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.166+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f6 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041.out auth v164 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f6:[<0x100000003a4/041.out v164>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f6:[<0x100000003a4/041.out v164>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.166+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e8 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034.out auth v136 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e8:[<0x100000003a4/034.out v136>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e8:[<0x100000003a4/034.out v136>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.166+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ee [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037.out auth v148 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ee:[<0x100000003a4/037.out v148>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ee:[<0x100000003a4/037.out v148>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.166+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e1 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031 auth v122 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e1:[<0x100000003a4/031 v122>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e1:[<0x100000003a4/031 v122>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.251 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.167+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003db [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028 auth v110 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003db:[<0x100000003a4/028 v110>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003db:[<0x100000003a4/028 v110>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.167+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c8 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018.out auth v72 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c8:[<0x100000003a4/018.out v72>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c8:[<0x100000003a4/018.out v72>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.167+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e7 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034 auth v134 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e7:[<0x100000003a4/034 v134>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e7:[<0x100000003a4/034 v134>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.168+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ea [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035.out auth v140 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ea:[<0x100000003a4/035.out v140>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ea:[<0x100000003a4/035.out v140>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.168+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000400 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046.out auth v184 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000400:[<0x100000003a4/046.out v184>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000400:[<0x100000003a4/046.out v184>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.168+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fa [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043.out auth v172 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003fa:[<0x100000003a4/043.out v172>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003fa:[<0x100000003a4/043.out v172>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.168+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cb [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020 auth v78 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003cb:[<0x100000003a4/020 v78>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003cb:[<0x100000003a4/020 v78>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.168+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f3 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040 auth v158 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f3:[<0x100000003a4/040 v158>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f3:[<0x100000003a4/040 v158>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.168+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f4 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040.out auth v160 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4534000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f4:[<0x100000003a4/040.out v160>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f4:[<0x100000003a4/040.out v160>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.169+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b4 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008.out auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4535080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b4:[<0x100000003a4/008.out v32>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b4:[<0x100000003a4/008.out v32>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.169+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bb [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012 auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d452f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bb:[<0x100000003a4/012 v46>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003bb:[<0x100000003a4/012 v46>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.169+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ba [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011.out auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4535600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ba:[<0x100000003a4/011.out v44>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ba:[<0x100000003a4/011.out v44>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.252 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.169+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c3 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016 auth v62 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4534580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c3:[<0x100000003a4/016 v62>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c3:[<0x100000003a4/016 v62>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.170+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004.out auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4534b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ac:[<0x100000003a4/004.out v16>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ac:[<0x100000003a4/004.out v16>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.170+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a7 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002 auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4535b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a7:[<0x100000003a4/002 v6>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003a7:[<0x100000003a4/002 v6>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.170+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ae [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005.out auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4536100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ae:[<0x100000003a4/005.out v20>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ae:[<0x100000003a4/005.out v20>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.171+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ca [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019.out auth v76 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4536680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ca:[<0x100000003a4/019.out v76>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ca:[<0x100000003a4/019.out v76>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.171+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000407 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050 auth v198 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4537700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000407:[<0x100000003a4/050 v198>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000407:[<0x100000003a4/050 v198>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.171+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fc [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044.out auth v176 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d453e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003fc:[<0x100000003a4/044.out v176>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003fc:[<0x100000003a4/044.out v176>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.171+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043e [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077.out auth v304 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4536c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000043e:[<0x100000003a4/077.out v304>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000043e:[<0x100000003a4/077.out v304>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.171+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000436 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073.out auth v292 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4537180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000436:[<0x100000003a4/073.out v292>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000436:[<0x100000003a4/073.out v292>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.253 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.172+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003df [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030 auth v118 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d453e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003df:[<0x100000003a4/030 v118>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003df:[<0x100000003a4/030 v118>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bc [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012.out auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d453f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bc:[<0x100000003a4/012.out v48>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003bc:[<0x100000003a4/012.out v48>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003aa [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003.out auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d453f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003aa:[<0x100000003a4/003.out v12>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003aa:[<0x100000003a4/003.out v12>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f7 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042 auth v166 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d453eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f7:[<0x100000003a4/042 v166>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f7:[<0x100000003a4/042 v166>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000431 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071 auth v282 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d453fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000431:[<0x100000003a4/071 v282>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000431:[<0x100000003a4/071 v282>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000424 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064.out auth v256 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4540100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000424:[<0x100000003a4/064.out v256>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000424:[<0x100000003a4/064.out v256>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b1 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007 auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4541180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b1:[<0x100000003a4/007 v26>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b1:[<0x100000003a4/007 v26>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.173+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e3 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032 auth v126 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4541700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e3:[<0x100000003a4/032 v126>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e3:[<0x100000003a4/032 v126>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.174+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d1 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023 auth v90 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4540c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d1:[<0x100000003a4/023 v90>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d1:[<0x100000003a4/023 v90>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.174+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c4 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016.out auth v64 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4540680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c4:[<0x100000003a4/016.out v64>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c4:[<0x100000003a4/016.out v64>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.174+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ff [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046 auth v182 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4568580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ff:[<0x100000003a4/046 v182>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ff:[<0x100000003a4/046 v182>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.174+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b5 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009 auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4568b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b5:[<0x100000003a4/009 v34>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b5:[<0x100000003a4/009 v34>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.259 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.175+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e5 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033 auth v130 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4568000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e5:[<0x100000003a4/033 v130>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e5:[<0x100000003a4/033 v130>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.175+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000432 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071.out auth v284 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4569080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000432:[<0x100000003a4/071.out v284>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000432:[<0x100000003a4/071.out v284>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.175+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fe [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045.out auth v180 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4569600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003fe:[<0x100000003a4/045.out v180>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003fe:[<0x100000003a4/045.out v180>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.175+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b8 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010.out auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d456a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b8:[<0x100000003a4/010.out v40>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b8:[<0x100000003a4/010.out v40>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.175+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043b [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076 auth v302 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d456a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000043b:[<0x100000003a4/076 v302>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000043b:[<0x100000003a4/076 v302>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.175+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d6 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025.out auth v100 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d456b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d6:[<0x100000003a4/025.out v100>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d6:[<0x100000003a4/025.out v100>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.176+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bd [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013 auth v50 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d456b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bd:[<0x100000003a4/013 v50>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003bd:[<0x100000003a4/013 v50>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.176+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041c [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060.out auth v240 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4574000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000041c:[<0x100000003a4/060.out v240>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000041c:[<0x100000003a4/060.out v240>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.177+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d8 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026.out auth v104 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d456ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d8:[<0x100000003a4/026.out v104>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d8:[<0x100000003a4/026.out v104>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.177+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000404 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048.out auth v192 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4569b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000404:[<0x100000003a4/048.out v192>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000404:[<0x100000003a4/048.out v192>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.177+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000423 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064 auth v254 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4574b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000423:[<0x100000003a4/064 v254>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000423:[<0x100000003a4/064 v254>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.177+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043c [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076.out auth v304 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4575600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000043c:[<0x100000003a4/076.out v304>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000043c:[<0x100000003a4/076.out v304>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.260 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.177+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000413 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056 auth v222 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4575b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000413:[<0x100000003a4/056 v222>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000413:[<0x100000003a4/056 v222>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.178+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f8 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042.out auth v168 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4576680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f8:[<0x100000003a4/042.out v168>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f8:[<0x100000003a4/042.out v168>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.178+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c0 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014.out auth v56 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4576c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c0:[<0x100000003a4/014.out v56>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c0:[<0x100000003a4/014.out v56>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.178+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d7 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026 auth v102 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4575080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d7:[<0x100000003a4/026 v102>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d7:[<0x100000003a4/026 v102>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.178+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c1 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015 auth v58 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4574580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c1:[<0x100000003a4/015 v58>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c1:[<0x100000003a4/015 v58>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.178+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040c [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052.out auth v208 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4576100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040c:[<0x100000003a4/052.out v208>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000040c:[<0x100000003a4/052.out v208>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.179+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000430 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070.out auth v280 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d457eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000430:[<0x100000003a4/070.out v280>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000430:[<0x100000003a4/070.out v280>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.179+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a5 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001 auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d457f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a5:[<0x100000003a4/001 v2>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003a5:[<0x100000003a4/001 v2>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.180+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d2 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023.out auth v92 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4577700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d2:[<0x100000003a4/023.out v92>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d2:[<0x100000003a4/023.out v92>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.180+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d4 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024.out auth v96 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4577180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d4:[<0x100000003a4/024.out v96>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d4:[<0x100000003a4/024.out v96>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.180+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042b [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068 auth v270 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d457e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000042b:[<0x100000003a4/068 v270>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000042b:[<0x100000003a4/068 v270>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.180+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000409 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051 auth v202 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d457e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000409:[<0x100000003a4/051 v202>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000409:[<0x100000003a4/051 v202>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.180+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b3 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008 auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d457f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b3:[<0x100000003a4/008 v30>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b3:[<0x100000003a4/008 v30>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.261 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.180+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b2 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007.out auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d457fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b2:[<0x100000003a4/007.out v28>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b2:[<0x100000003a4/007.out v28>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.181+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042c [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068.out auth v272 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4580680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000042c:[<0x100000003a4/068.out v272>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000042c:[<0x100000003a4/068.out v272>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.181+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d3 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024 auth v94 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4580c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d3:[<0x100000003a4/024 v94>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d3:[<0x100000003a4/024 v94>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.181+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000421 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063 auth v250 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4581180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000421:[<0x100000003a4/063 v250>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000421:[<0x100000003a4/063 v250>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.181+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000420 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062.out auth v248 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4581700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000420:[<0x100000003a4/062.out v248>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000420:[<0x100000003a4/062.out v248>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.182+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000411 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055 auth v218 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4588000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000411:[<0x100000003a4/055 v218>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000411:[<0x100000003a4/055 v218>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.182+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000410 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054.out auth v216 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4589600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000410:[<0x100000003a4/054.out v216>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000410:[<0x100000003a4/054.out v216>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.182+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040e [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053.out auth v212 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4589b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040e:[<0x100000003a4/053.out v212>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000040e:[<0x100000003a4/053.out v212>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.183+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000406 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049.out auth v196 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4588b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000406:[<0x100000003a4/049.out v196>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000406:[<0x100000003a4/049.out v196>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.183+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000428 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066.out auth v264 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4588580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000428:[<0x100000003a4/066.out v264>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000428:[<0x100000003a4/066.out v264>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.183+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fd [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045 auth v178 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d458a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003fd:[<0x100000003a4/045 v178>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003fd:[<0x100000003a4/045 v178>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.183+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041e [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061.out auth v244 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d458ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000041e:[<0x100000003a4/061.out v244>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000041e:[<0x100000003a4/061.out v244>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.184+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d5 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025 auth v98 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4596580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d5:[<0x100000003a4/025 v98>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d5:[<0x100000003a4/025 v98>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.184+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b7 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010 auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d458b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b7:[<0x100000003a4/010 v38>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b7:[<0x100000003a4/010 v38>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.262 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.184+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041b [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060 auth v238 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4596b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000041b:[<0x100000003a4/060 v238>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000041b:[<0x100000003a4/060 v238>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.185+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b9 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011 auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d458b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b9:[<0x100000003a4/011 v42>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b9:[<0x100000003a4/011 v42>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.185+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a8 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002.out auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4597600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a8:[<0x100000003a4/002.out v8>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003a8:[<0x100000003a4/002.out v8>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.185+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000433 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072 auth v286 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4597080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000433:[<0x100000003a4/072 v286>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000433:[<0x100000003a4/072 v286>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.185+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ec [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036.out auth v144 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4597b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ec:[<0x100000003a4/036.out v144>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ec:[<0x100000003a4/036.out v144>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.186+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040f [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054 auth v214 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4598100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040f:[<0x100000003a4/054 v214>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000040f:[<0x100000003a4/054 v214>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.186+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000417 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058 auth v230 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4598680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000417:[<0x100000003a4/058 v230>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000417:[<0x100000003a4/058 v230>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.186+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040b [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052 auth v206 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4598c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040b:[<0x100000003a4/052 v206>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000040b:[<0x100000003a4/052 v206>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.186+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f5 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041 auth v162 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4599700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f5:[<0x100000003a4/041 v162>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f5:[<0x100000003a4/041 v162>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.186+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d0 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022.out auth v88 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d4599180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d0:[<0x100000003a4/022.out v88>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d0:[<0x100000003a4/022.out v88>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.186+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b6 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009.out auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d459e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b6:[<0x100000003a4/009.out v36>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b6:[<0x100000003a4/009.out v36>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.187+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b0 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006.out auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d459e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b0:[<0x100000003a4/006.out v24>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003b0:[<0x100000003a4/006.out v24>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.263 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.188+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043d [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077 auth v304 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d459eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000043d:[<0x100000003a4/077 v304>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000043d:[<0x100000003a4/077 v304>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.188+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cd [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021 auth v82 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d459f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003cd:[<0x100000003a4/021 v82>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003cd:[<0x100000003a4/021 v82>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.188+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ad [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005 auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d459f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ad:[<0x100000003a4/005 v18>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ad:[<0x100000003a4/005 v18>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.188+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040d [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053 auth v210 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d459fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040d:[<0x100000003a4/053 v210>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000040d:[<0x100000003a4/053 v210>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.189+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003de [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029.out auth v116 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003de:[<0x100000003a4/029.out v116>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003de:[<0x100000003a4/029.out v116>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.189+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000402 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047.out auth v188 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000402:[<0x100000003a4/047.out v188>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000402:[<0x100000003a4/047.out v188>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.189+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c6 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017.out auth v68 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c6:[<0x100000003a4/017.out v68>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c6:[<0x100000003a4/017.out v68>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.189+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000401 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047 auth v186 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000401:[<0x100000003a4/047 v186>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000401:[<0x100000003a4/047 v186>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.190+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000418 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058.out auth v232 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000418:[<0x100000003a4/058.out v232>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000418:[<0x100000003a4/058.out v232>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.191+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018 auth v70 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c7:[<0x100000003a4/018 v70>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c7:[<0x100000003a4/018 v70>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.191+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e9 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035 auth v138 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e9:[<0x100000003a4/035 v138>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e9:[<0x100000003a4/035 v138>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.191+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bf [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014 auth v54 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bf:[<0x100000003a4/014 v54>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003bf:[<0x100000003a4/014 v54>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.191+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041f [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062 auth v246 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45aa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000041f:[<0x100000003a4/062 v246>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000041f:[<0x100000003a4/062 v246>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.264 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.191+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fb [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044 auth v174 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45a9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003fb:[<0x100000003a4/044 v174>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003fb:[<0x100000003a4/044 v174>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.192+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ce [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021.out auth v84 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45aac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ce:[<0x100000003a4/021.out v84>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ce:[<0x100000003a4/021.out v84>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.192+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e2 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031.out auth v124 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45aa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e2:[<0x100000003a4/031.out v124>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e2:[<0x100000003a4/031.out v124>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.192+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c2 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015.out auth v60 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45ab180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c2:[<0x100000003a4/015.out v60>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c2:[<0x100000003a4/015.out v60>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.193+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f2 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039.out auth v156 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f2:[<0x100000003a4/039.out v156>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f2:[<0x100000003a4/039.out v156>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.193+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000426 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065.out auth v260 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45ab700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000426:[<0x100000003a4/065.out v260>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000426:[<0x100000003a4/065.out v260>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.193+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000405 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049 auth v194 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000405:[<0x100000003a4/049 v194>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000405:[<0x100000003a4/049 v194>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.193+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e0 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030.out auth v120 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e0:[<0x100000003a4/030.out v120>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e0:[<0x100000003a4/030.out v120>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.193+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f0 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038.out auth v152 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f0:[<0x100000003a4/038.out v152>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f0:[<0x100000003a4/038.out v152>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.193+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017 auth v66 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c5:[<0x100000003a4/017 v66>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c5:[<0x100000003a4/017 v66>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.194+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000419 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059 auth v234 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000419:[<0x100000003a4/059 v234>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000419:[<0x100000003a4/059 v234>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.194+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042f [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070 auth v278 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000042f:[<0x100000003a4/070 v278>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000042f:[<0x100000003a4/070 v278>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.195+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042e [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069.out auth v276 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000042e:[<0x100000003a4/069.out v276>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000042e:[<0x100000003a4/069.out v276>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.265 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.195+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dd [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029 auth v114 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003dd:[<0x100000003a4/029 v114>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003dd:[<0x100000003a4/029 v114>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.195+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000425 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065 auth v258 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000425:[<0x100000003a4/065 v258>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000425:[<0x100000003a4/065 v258>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.195+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003af [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006 auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45be000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003af:[<0x100000003a4/006 v22>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003af:[<0x100000003a4/006 v22>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.196+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000438 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074.out auth v296 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45beb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000438:[<0x100000003a4/074.out v296>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000438:[<0x100000003a4/074.out v296>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.196+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042d [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069 auth v274 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45bf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000042d:[<0x100000003a4/069 v274>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000042d:[<0x100000003a4/069 v274>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.196+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cc [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020.out auth v80 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45b5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003cc:[<0x100000003a4/020.out v80>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003cc:[<0x100000003a4/020.out v80>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.197+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040a [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051.out auth v204 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45bf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040a:[<0x100000003a4/051.out v204>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000040a:[<0x100000003a4/051.out v204>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.197+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000422 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063.out auth v252 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000422:[<0x100000003a4/063.out v252>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000422:[<0x100000003a4/063.out v252>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.197+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c9 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019 auth v74 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c9:[<0x100000003a4/019 v74>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003c9:[<0x100000003a4/019 v74>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.198+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000414 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056.out auth v224 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000414:[<0x100000003a4/056.out v224>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000414:[<0x100000003a4/056.out v224>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.198+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003da [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027.out auth v108 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45bfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003da:[<0x100000003a4/027.out v108>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003da:[<0x100000003a4/027.out v108>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.198+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ef [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038 auth v150 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ef:[<0x100000003a4/038 v150>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ef:[<0x100000003a4/038 v150>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.199+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cf [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022 auth v86 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003cf:[<0x100000003a4/022 v86>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003cf:[<0x100000003a4/022 v86>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.266 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.199+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000437 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074 auth v294 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000437:[<0x100000003a4/074 v294>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000437:[<0x100000003a4/074 v294>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.199+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000403 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048 auth v190 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000403:[<0x100000003a4/048 v190>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000403:[<0x100000003a4/048 v190>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.199+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003be [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013.out auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003be:[<0x100000003a4/013.out v52>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003be:[<0x100000003a4/013.out v52>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.199+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003eb [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036 auth v142 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003eb:[<0x100000003a4/036 v142>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003eb:[<0x100000003a4/036 v142>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.199+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000434 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072.out auth v288 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000434:[<0x100000003a4/072.out v288>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000434:[<0x100000003a4/072.out v288>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.200+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000429 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067 auth v266 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45ca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000429:[<0x100000003a4/067 v266>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000429:[<0x100000003a4/067 v266>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.200+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000412 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055.out auth v220 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45ca680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000412:[<0x100000003a4/055.out v220>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000412:[<0x100000003a4/055.out v220>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.200+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041d [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061 auth v242 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45c9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000041d:[<0x100000003a4/061 v242>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000041d:[<0x100000003a4/061 v242>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.200+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000415 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057 auth v226 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45cac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000415:[<0x100000003a4/057 v226>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000415:[<0x100000003a4/057 v226>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.201+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000416 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057.out auth v228 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45cb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000416:[<0x100000003a4/057.out v228>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000416:[<0x100000003a4/057.out v228>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.201+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000408 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050.out auth v200 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000408:[<0x100000003a4/050.out v200>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000408:[<0x100000003a4/050.out v200>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.201+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ab [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004 auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ab:[<0x100000003a4/004 v14>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ab:[<0x100000003a4/004 v14>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.201+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a6 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001.out auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a6:[<0x100000003a4/001.out v4>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003a6:[<0x100000003a4/001.out v4>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.201+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d9 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027 auth v106 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d9:[<0x100000003a4/027 v106>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003d9:[<0x100000003a4/027 v106>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.267 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.201+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f9 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043 auth v170 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f9:[<0x100000003a4/043 v170>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f9:[<0x100000003a4/043 v170>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.268 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.202+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f1 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039 auth v154 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003f1:[<0x100000003a4/039 v154>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003f1:[<0x100000003a4/039 v154>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.268 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.202+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e4 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032.out auth v128 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e4:[<0x100000003a4/032.out v128>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e4:[<0x100000003a4/032.out v128>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.268 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.203+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000439 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075 auth v298 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000439:[<0x100000003a4/075 v298>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000439:[<0x100000003a4/075 v298>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.268 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.203+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000435 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073 auth v290 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45dc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000435:[<0x100000003a4/073 v290>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000435:[<0x100000003a4/073 v290>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.203+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e6 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033.out auth v132 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003e6:[<0x100000003a4/033.out v132>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003e6:[<0x100000003a4/033.out v132>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.203+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043a [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075.out auth v300 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45d5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000043a:[<0x100000003a4/075.out v300>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000043a:[<0x100000003a4/075.out v300>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.203+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dc [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028.out auth v112 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45dc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003dc:[<0x100000003a4/028.out v112>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003dc:[<0x100000003a4/028.out v112>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.203+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ed [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037 auth v146 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45dcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ed:[<0x100000003a4/037 v146>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x100000003ed:[<0x100000003a4/037 v146>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.204+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041a [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059.out auth v236 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45dd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000041a:[<0x100000003a4/059.out v236>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000041a:[<0x100000003a4/059.out v236>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:29:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:29:06.205+0000 7fc6a0d66700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000427 [3,head] /volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066 auth v262 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25277=0-4194304@2} caps={25277=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25277 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5599d45dd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000427:[<0x100000003a4/066 v262>,<0x100000003a2/btrfs v320>,<0x10000000007/tests v772>,<0x10000000006/xfstests-dev v74>,<0x10000000005/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/qa v36>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000427:[<0x100000003a4/066 v262>,<0x100000003a2/btrfs v338>,<0x10000000007/tests v784>,<0x10000000006/xfstests-dev v79>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v12>,<0x100000001fa/838d5287-f0ed-4bca-aa45-4ad0fce85018 v24>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v28>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T06:29:06.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:05 smithi090 ceph-mon[35681]: pgmap v32: 97 pgs: 97 active+clean; 16 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-02-26T06:29:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:05 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:05 smithi006 ceph-mon[31790]: pgmap v32: 97 pgs: 97 active+clean; 16 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-02-26T06:29:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:05 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: scrub summary: active paths [/] 2024-02-26T06:29:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000006 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/MAINTAINERS) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002d9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/src) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000028d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/doc) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000249 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.selftest) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002a6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lib) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000294 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/include) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000245 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.config-sections) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000024b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/acinclude.m4) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002d4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/new) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makefile) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002c1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/m4) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002d5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/randomize.awk) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000289 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/crash) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002b7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ltp) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002b5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/local.config.example) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makepkgs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000247 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.fuse) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002b6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lsqa.pl) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000244 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002d6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/release.sh) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000246 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.device-mapper) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configure.ac) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.git) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000255 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/common) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002d8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/soak) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/LICENSES) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000248 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.overlay) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000024a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/VERSION) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.gitignore) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000254 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/check) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000002d7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/setup) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000024c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/build) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: scrub summary: active paths [/] 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/Makefile) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000042a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ee (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003db (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000400 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003fa (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000407 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003fc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000043e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000436 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003df (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000431 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.371 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ff (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000432 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003fe (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000043b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000041c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000404 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000423 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000043c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.372 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000413 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000040c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000430 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000409 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000420 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000411 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000410 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000040e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000406 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000428 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003fd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000041e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000041b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000433 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ec (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000040f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000417 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000040b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000043d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000040d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003de (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000402 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000401 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000418 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000041f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003fb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000405 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000419 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000042f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000042e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000425 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003af (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000438 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000042d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000040a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000422 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000414 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003da (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000437 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000403 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003be (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000434 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000429 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000412 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000041d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000415 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000416 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000408 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.378 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000439 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000435 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000043a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x100000003ed (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x1000000041a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: Scrub error on inode 0x10000000427 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:07 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: scrub summary: active paths [/] 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000006 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/MAINTAINERS) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002d9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/src) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000028d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/doc) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000249 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.selftest) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002a6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lib) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000294 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/include) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000245 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.config-sections) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000024b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/acinclude.m4) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002d4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/new) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makefile) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002c1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/m4) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002d5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/randomize.awk) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000289 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/crash) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002b7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ltp) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002b5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/local.config.example) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makepkgs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000247 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.fuse) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002b6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lsqa.pl) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000244 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002d6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/release.sh) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000246 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.device-mapper) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configure.ac) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.git) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000255 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/common) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002d8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/soak) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/LICENSES) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000248 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.overlay) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000024a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/VERSION) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.gitignore) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000254 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/check) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000002d7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/setup) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000024c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/build) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: scrub summary: active paths [/] 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/Makefile) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000042a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ee (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003db (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000400 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003fa (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.426 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000407 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003fc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000043e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000436 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.427 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003df (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000431 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ff (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000432 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003fe (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000043b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000041c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000404 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000423 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000043c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000413 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000040c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000430 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000409 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000420 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000411 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000410 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000040e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000406 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000428 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003fd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000041e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.430 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000041b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000433 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ec (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000040f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000417 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000040b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000043d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000040d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003de (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000402 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000401 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000418 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000041f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003fb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000405 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000419 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000042f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.433 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000042e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000425 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003af (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000438 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000042d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000040a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000422 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000414 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003da (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000437 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000403 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.434 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003be (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000434 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000429 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000412 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000041d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000415 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000416 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000408 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000439 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000435 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.435 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.436 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000043a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.436 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.436 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x100000003ed (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.436 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x1000000041a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.436 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: Scrub error on inode 0x10000000427 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.436 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:07 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: scrub summary: active paths [/] 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000006 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/MAINTAINERS) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002d9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/src) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000028d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/doc) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000249 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.selftest) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002a6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lib) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000294 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/include) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000245 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.config-sections) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000024b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/acinclude.m4) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002d4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/new) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makefile) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002c1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/m4) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002d5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/randomize.awk) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000289 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/crash) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002b7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ltp) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002b5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/local.config.example) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makepkgs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000247 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.fuse) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002b6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lsqa.pl) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000244 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002d6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/release.sh) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000246 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.device-mapper) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configure.ac) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.git) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000255 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/common) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002d8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/soak) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/LICENSES) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000248 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.overlay) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000024a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/VERSION) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.gitignore) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000254 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/check) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000002d7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/setup) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000024c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/build) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: scrub summary: active paths [/] 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/Makefile) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000042a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ee (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003db (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000400 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003fa (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000407 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003fc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000043e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000436 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003df (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000431 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ff (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000432 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003fe (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000043b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000041c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000404 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000423 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000043c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000413 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000040c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000430 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000409 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000420 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000411 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000410 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000040e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000406 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000428 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003fd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000041e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000041b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000433 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ec (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000040f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000417 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000040b (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000043d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000040d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003de (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000402 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000401 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000418 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000041f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003fb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000405 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000419 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000042f (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000042e (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000425 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003af (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000438 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000042d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000040a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000422 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000414 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003da (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000437 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000403 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003be (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000434 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000429 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000412 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000041d (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000415 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000416 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000408 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f9 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000439 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000435 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000043a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x100000003ed (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x1000000041a (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: Scrub error on inode 0x10000000427 (/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.i log and `damage ls` output for details 2024-02-26T06:29:07.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:07 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:08 smithi090 ceph-mon[35681]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:08 smithi090 ceph-mon[35681]: pgmap v33: 97 pgs: 97 active+clean; 16 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-02-26T06:29:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:08 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:08 smithi006 ceph-mon[31790]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:08 smithi006 ceph-mon[31790]: pgmap v33: 97 pgs: 97 active+clean; 16 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-02-26T06:29:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:08 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:08 smithi167 ceph-mon[36229]: scrub summary: idle+waiting paths [/] 2024-02-26T06:29:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:08 smithi167 ceph-mon[36229]: pgmap v33: 97 pgs: 97 active+clean; 16 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-02-26T06:29:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:08 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:10 smithi090 ceph-mon[35681]: pgmap v34: 97 pgs: 97 active+clean; 17 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-02-26T06:29:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:10 smithi006 ceph-mon[31790]: pgmap v34: 97 pgs: 97 active+clean; 17 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-02-26T06:29:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:10 smithi167 ceph-mon[36229]: pgmap v34: 97 pgs: 97 active+clean; 17 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-02-26T06:29:11.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:11 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:11.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:29:11.023+0000 7efe1ac00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:11.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:11 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:11.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:11 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:12 smithi090 ceph-mon[35681]: pgmap v35: 97 pgs: 97 active+clean; 17 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-02-26T06:29:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:12 smithi090 ceph-mon[35681]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:12 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:active 2024-02-26T06:29:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:12 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:12 smithi006 ceph-mon[31790]: pgmap v35: 97 pgs: 97 active+clean; 17 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-02-26T06:29:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:12 smithi006 ceph-mon[31790]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:12 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:active 2024-02-26T06:29:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:12 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:12 smithi167 ceph-mon[36229]: pgmap v35: 97 pgs: 97 active+clean; 17 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-02-26T06:29:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:12 smithi167 ceph-mon[36229]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T06:29:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:12 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] up:active 2024-02-26T06:29:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:12 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:14 smithi090 ceph-mon[35681]: pgmap v36: 97 pgs: 97 active+clean; 20 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-02-26T06:29:14.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:14 smithi006 ceph-mon[31790]: pgmap v36: 97 pgs: 97 active+clean; 20 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-02-26T06:29:14.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:14 smithi167 ceph-mon[36229]: pgmap v36: 97 pgs: 97 active+clean; 20 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-02-26T06:29:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 -- 172.21.15.6:0/4111267391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec4103f80 msgr2=0x7f7ec4107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 --2- 172.21.15.6:0/4111267391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec4103f80 0x7f7ec4107fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009930 tx=0x7f7eb002bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 -- 172.21.15.6:0/4111267391 shutdown_connections 2024-02-26T06:29:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 --2- 172.21.15.6:0/4111267391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec41086a0 0x7f7ec4110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 --2- 172.21.15.6:0/4111267391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec4103f80 0x7f7ec4107fc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 --2- 172.21.15.6:0/4111267391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41035d0 0x7f7ec41039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 -- 172.21.15.6:0/4111267391 >> 172.21.15.6:0/4111267391 conn(0x7f7ec40fee70 msgr2=0x7f7ec4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 -- 172.21.15.6:0/4111267391 shutdown_connections 2024-02-26T06:29:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.286+0000 7f7ec9800640 1 -- 172.21.15.6:0/4111267391 wait complete. 2024-02-26T06:29:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.287+0000 7f7ec9800640 1 Processor -- start 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.287+0000 7f7ec9800640 1 -- start start 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.287+0000 7f7ec9800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec41035d0 0x7f7ec419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.287+0000 7f7ec9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 0x7f7ec419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.287+0000 7f7ec9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec41998d0 0x7f7ec4199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec9800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419a2c0 con 0x7f7ec41035d0 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419a430 con 0x7f7ec41998d0 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419a5f0 con 0x7f7ec41086a0 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 0x7f7ec419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec41035d0 0x7f7ec419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 0x7f7ec419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34354/0 (socket says 172.21.15.6:34354) 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 -- 172.21.15.6:0/1109827061 learned_addr learned my addr 172.21.15.6:0/1109827061 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec3400640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec41998d0 0x7f7ec4199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 -- 172.21.15.6:0/1109827061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec41998d0 msgr2=0x7f7ec4199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec41998d0 0x7f7ec4199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 -- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec41035d0 msgr2=0x7f7ec419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec41035d0 0x7f7ec419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.288+0000 7f7ec2000640 1 -- 172.21.15.6:0/1109827061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0009560 con 0x7f7ec41086a0 2024-02-26T06:29:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.289+0000 7f7ec2000640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 0x7f7ec419fc10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005ab0 tx=0x7f7eb002c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.289+0000 7f7ebb400640 1 -- 172.21.15.6:0/1109827061 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00042d0 con 0x7f7ec41086a0 2024-02-26T06:29:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.289+0000 7f7ebb400640 1 -- 172.21.15.6:0/1109827061 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0004430 con 0x7f7ec41086a0 2024-02-26T06:29:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.289+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec41a6590 con 0x7f7ec41086a0 2024-02-26T06:29:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.289+0000 7f7ebb400640 1 -- 172.21.15.6:0/1109827061 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0003be0 con 0x7f7ec41086a0 2024-02-26T06:29:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.289+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41a6a60 con 0x7f7ec41086a0 2024-02-26T06:29:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.292+0000 7f7ecc83a740 1 Processor -- start 2024-02-26T06:29:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.292+0000 7f7ebb400640 1 -- 172.21.15.6:0/1109827061 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f7eb0003d40 con 0x7f7ec41086a0 2024-02-26T06:29:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.292+0000 7f7ecc83a740 1 -- start start 2024-02-26T06:29:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bceab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bceac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bce675f0 con 0x55d7bceabd80 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bce19890 con 0x55d7bceb5300 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bce16dc0 con 0x55d7bceacaf0 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bceab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bceab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34364/0 (socket says 172.21.15.6:34364) 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ec3400640 1 -- 172.21.15.6:0/3286070021 learned_addr learned my addr 172.21.15.6:0/3286070021 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.293+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bceac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ec3400640 1 -- 172.21.15.6:0/3286070021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 msgr2=0x55d7bceb76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ec3400640 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ec3400640 1 -- 172.21.15.6:0/3286070021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 msgr2=0x55d7bceac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ec3400640 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bceac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ec3400640 1 -- 172.21.15.6:0/3286070021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7bcd93ad0 con 0x55d7bceacaf0 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ec3400640 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bceab840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009f50 tx=0x7f7ebc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb8c183966f92e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ebb400640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7ea80a23c0 0x7f7ea80a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7eafe00640 1 -- 172.21.15.6:0/3286070021 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc004d20 con 0x55d7bceacaf0 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7eafe00640 1 -- 172.21.15.6:0/3286070021 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc004e80 con 0x55d7bceacaf0 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7eafe00640 1 -- 172.21.15.6:0/3286070021 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc005280 con 0x55d7bceacaf0 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.294+0000 7f7ebb400640 1 -- 172.21.15.6:0/1109827061 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7eb00f0870 con 0x7f7ec41086a0 2024-02-26T06:29:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7ea80a23c0 0x7f7ea80a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3286070021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 msgr2=0x55d7bceab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bceab840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009f50 tx=0x7f7ebc0118d0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3286070021 shutdown_connections 2024-02-26T06:29:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bceac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3286070021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bceab840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3286070021 >> 172.21.15.6:0/3286070021 conn(0x55d7bce73480 msgr2=0x55d7bcea6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3286070021 shutdown_connections 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7ea80a23c0 0x7f7ea80a4880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55d7bcead250 tx=0x7f7eb4005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.295+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3286070021 wait complete. 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.296+0000 7f7ecc83a740 1 -- start start 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.296+0000 7f7ecc83a740 1 Processor -- start 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceb5300 0x55d7bceb2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bce125a0 con 0x55d7bceb5300 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcdb7b20 con 0x55d7bceacaf0 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcdb77c0 con 0x55d7bceabd80 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ecc83a740 10 client.0 _ll_register_callbacks cb 0x55d7bcdcb520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34366/0 (socket says 172.21.15.6:34366) 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3616163508 learned_addr learned my addr 172.21.15.6:0/3616163508 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec3400640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceb5300 0x55d7bceb2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceacaf0 msgr2=0x55d7bceb2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3616163508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceb5300 msgr2=0x55d7bceb2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceb5300 0x55d7bceb2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3616163508 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc009c00 con 0x55d7bceabd80 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.297+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb2330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4004650 tx=0x7f7eb4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72895f36677026fc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7eacc00640 1 -- 172.21.15.6:0/3616163508 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4023070 con 0x55d7bceabd80 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7bcde4ca0 con 0x55d7bceabd80 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7eacc00640 1 -- 172.21.15.6:0/3616163508 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7eb4029420 con 0x55d7bceabd80 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7eacc00640 1 -- 172.21.15.6:0/3616163508 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eb4017070 con 0x55d7bceabd80 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7ecc83a740 10 client.25897 fetch_fsmap learned FSMap version 33 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d7bce10c20 con 0x55d7bceabd80 2024-02-26T06:29:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.298+0000 7f7eacc00640 1 -- 172.21.15.6:0/3616163508 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb4036080 con 0x55d7bceabd80 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7eacc00640 1 -- 172.21.15.6:0/3616163508 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7eb400f580 con 0x55d7bceabd80 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ecc83a740 10 client.25897 fetch_fsmap finished waiting for FSMap version 33 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ecc83a740 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ecc83a740 20 client.25897 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcec5450 0x55d7bcec7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ecc83a740 4 client.25897 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7bce08cb0 con 0x55d7bcec5450 2024-02-26T06:29:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.299+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcec5450 0x55d7bcec7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.300+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcec5450 0x55d7bcec7890 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4dc04cfde129ad5 server_cookie=ac208bce68f8a52 in_seq=0 out_seq=0 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.300+0000 7f7eacc00640 10 client.25897 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.300+0000 7f7eacc00640 1 -- 172.21.15.6:0/3616163508 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7bce08cb0 con 0x55d7bcec5450 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.300+0000 7f7eacc00640 10 client.25897 handle_command_reply: tid=0 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.300+0000 7f7ecc83a740 1 client.25897 shutdown 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 msgr2=0x55d7bceb2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb2330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4004650 tx=0x7f7eb4004730 comp rx=0 tx=0).stop 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 shutdown_connections 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcec5450 0x55d7bcec7890 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceb5300 0x55d7bceb2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7eacc00640 0 client.25897 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3616163508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb2330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 >> 172.21.15.6:0/3616163508 conn(0x55d7bce73480 msgr2=0x55d7bcea87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 shutdown_connections 2024-02-26T06:29:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3616163508 wait complete. 2024-02-26T06:29:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.301+0000 7f7ecc83a740 20 client.25897 trim_cache size 0 max 16384 2024-02-26T06:29:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 Processor -- start 2024-02-26T06:29:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 -- start start 2024-02-26T06:29:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcb5ae60 con 0x55d7bceacaf0 2024-02-26T06:29:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bceab340 con 0x55d7bceb5300 2024-02-26T06:29:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bceab050 con 0x55d7bceabd80 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.316+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34380/0 (socket says 172.21.15.6:34380) 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3183808706 learned_addr learned my addr 172.21.15.6:0/3183808706 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3183808706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 msgr2=0x55d7bceb2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3183808706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceacaf0 msgr2=0x55d7bceb2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 -- 172.21.15.6:0/3183808706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7bcdad500 con 0x55d7bceabd80 2024-02-26T06:29:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb4a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400f8b0 tx=0x7f7eb40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a63ba3b2921ab04 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7eaf400640 1 -- 172.21.15.6:0/3183808706 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4023070 con 0x55d7bceabd80 2024-02-26T06:29:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7eaf400640 1 -- 172.21.15.6:0/3183808706 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d7bcde4ca0 con 0x55d7bceabd80 2024-02-26T06:29:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.317+0000 7f7eaf400640 1 -- 172.21.15.6:0/3183808706 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400f5b0 con 0x55d7bceabd80 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3183808706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 msgr2=0x55d7bceb4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb4a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400f8b0 tx=0x7f7eb40046c0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3183808706 shutdown_connections 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bceb2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceacaf0 0x55d7bceb2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3183808706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceabd80 0x55d7bceb4a60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3183808706 >> 172.21.15.6:0/3183808706 conn(0x55d7bce73480 msgr2=0x55d7bceb0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3183808706 shutdown_connections 2024-02-26T06:29:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.318+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3183808706 wait complete. 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.319+0000 7f7ecc83a740 1 -- start start 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.319+0000 7f7ecc83a740 1 Processor -- start 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bcebad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bcebb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bcebb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcf1ba70 con 0x55d7bceabd80 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcf1bbe0 con 0x55d7bceb5300 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcf1bda0 con 0x55d7bceacaf0 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ecc83a740 10 client.0 _ll_register_callbacks cb 0x7f7eb4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bcebb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bcebb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:34388/0 (socket says 172.21.15.6:34388) 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 -- 172.21.15.6:0/3946357378 learned_addr learned my addr 172.21.15.6:0/3946357378 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec3400640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bcebb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bcebad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 -- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 msgr2=0x55d7bcebb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bcebb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 -- 172.21.15.6:0/3946357378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 msgr2=0x55d7bcebad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bcebad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.320+0000 7f7ec2000640 1 -- 172.21.15.6:0/3946357378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7bce10c20 con 0x55d7bceacaf0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ec2000640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bcebb2c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0004650 tx=0x7f7eb00f2c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398e368de9072d62 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7eacc00640 1 -- 172.21.15.6:0/3946357378 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00bd7a0 con 0x55d7bceacaf0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7eacc00640 1 -- 172.21.15.6:0/3946357378 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f7eb00bd0a0 con 0x55d7bceacaf0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7bce37d00 con 0x55d7bceacaf0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7eacc00640 1 -- 172.21.15.6:0/3946357378 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eb00c11f0 con 0x55d7bceacaf0 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 10 client.25903 fetch_fsmap learned FSMap version 33 2024-02-26T06:29:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d7bcdd3290 con 0x55d7bceacaf0 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7eacc00640 1 -- 172.21.15.6:0/3946357378 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb00d9080 con 0x55d7bceacaf0 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7eacc00640 1 -- 172.21.15.6:0/3946357378 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7eb003c070 con 0x55d7bceacaf0 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 10 client.25903 fetch_fsmap finished waiting for FSMap version 33 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 20 client.25903 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcf1d2b0 0x55d7bcf1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 4 client.25903 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.321+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7bcb12600 con 0x55d7bcf1d2b0 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.322+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcf1d2b0 0x55d7bcf1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.322+0000 7f7ec2a00640 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcf1d2b0 0x55d7bcf1f6d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ed3072b65b3263 server_cookie=51c58cdfba32b33a in_seq=0 out_seq=0 2024-02-26T06:29:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.322+0000 7f7eacc00640 10 client.25903 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7eacc00640 1 -- 172.21.15.6:0/3946357378 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7bcb12600 con 0x55d7bcf1d2b0 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7eacc00640 10 client.25903 handle_command_reply: tid=0 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 client.25903 shutdown 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 msgr2=0x55d7bcebb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bcebb2c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0004650 tx=0x7f7eb00f2c80 comp rx=0 tx=0).stop 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 shutdown_connections 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55d7bcf1d2b0 0x55d7bcf1f6d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d7bceb5300 0x55d7bcebb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7eacc00640 0 client.25903 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7bceacaf0 0x55d7bcebb2c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 --2- 172.21.15.6:0/3946357378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7bceabd80 0x55d7bcebad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 >> 172.21.15.6:0/3946357378 conn(0x55d7bce73480 msgr2=0x55d7bcea5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.331+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 shutdown_connections 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ecc83a740 1 -- 172.21.15.6:0/3946357378 wait complete. 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ecc83a740 20 client.25903 trim_cache size 0 max 16384 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:29:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:29:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7ea80a23c0 msgr2=0x7f7ea80a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7ea80a23c0 0x7f7ea80a4880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55d7bcead250 tx=0x7f7eb4005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 msgr2=0x7f7ec419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 0x7f7ec419fc10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005ab0 tx=0x7f7eb002c5f0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 shutdown_connections 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f7ea80a23c0 0x7f7ea80a4880 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec41998d0 0x7f7ec4199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec41086a0 0x7f7ec419fc10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.332+0000 7f7ec9800640 1 --2- 172.21.15.6:0/1109827061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ec41035d0 0x7f7ec419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.333+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 >> 172.21.15.6:0/1109827061 conn(0x7f7ec40fee70 msgr2=0x7f7ec4100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.333+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 shutdown_connections 2024-02-26T06:29:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:15.333+0000 7f7ec9800640 1 -- 172.21.15.6:0/1109827061 wait complete. 2024-02-26T06:29:15.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:15 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:29:15.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:15 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:29:15.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:15 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:29:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:16 smithi090 ceph-mon[35681]: pgmap v37: 97 pgs: 97 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-02-26T06:29:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:16 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:16 smithi006 ceph-mon[31790]: pgmap v37: 97 pgs: 97 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-02-26T06:29:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:16 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:16 smithi167 ceph-mon[36229]: pgmap v37: 97 pgs: 97 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-02-26T06:29:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:16 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:17 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:29:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:17 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:29:17.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:17 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:29:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:17 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:29:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:17 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:29:17.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:17 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:29:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:17 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:29:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:17 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:29:17.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:17 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:29:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:18 smithi090 ceph-mon[35681]: pgmap v38: 97 pgs: 97 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 493 KiB/s wr, 47 op/s 2024-02-26T06:29:18.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:18 smithi006 ceph-mon[31790]: pgmap v38: 97 pgs: 97 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 493 KiB/s wr, 47 op/s 2024-02-26T06:29:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:18 smithi167 ceph-mon[36229]: pgmap v38: 97 pgs: 97 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 493 KiB/s wr, 47 op/s 2024-02-26T06:29:19.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:19 smithi090 ceph-mon[35681]: pgmap v39: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 583 KiB/s wr, 60 op/s 2024-02-26T06:29:19.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:19 smithi006 ceph-mon[31790]: pgmap v39: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 583 KiB/s wr, 60 op/s 2024-02-26T06:29:19.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:19 smithi167 ceph-mon[36229]: pgmap v39: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 583 KiB/s wr, 60 op/s 2024-02-26T06:29:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:21 smithi090 ceph-mon[35681]: pgmap v40: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 475 KiB/s wr, 35 op/s 2024-02-26T06:29:21.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:21 smithi090 ceph-mon[35681]: scrub summary: idle 2024-02-26T06:29:21.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:21 smithi006 ceph-mon[31790]: pgmap v40: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 475 KiB/s wr, 35 op/s 2024-02-26T06:29:21.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:21 smithi006 ceph-mon[31790]: scrub summary: idle 2024-02-26T06:29:21.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:21 smithi167 ceph-mon[36229]: pgmap v40: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 475 KiB/s wr, 35 op/s 2024-02-26T06:29:21.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:21 smithi167 ceph-mon[36229]: scrub summary: idle 2024-02-26T06:29:22.268 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 11% (493/4260) Updating files: 11% (506/4260) Updating files: 12% (512/4260) Updating files: 13% (554/4260) Updating files: 14% (597/4260) Updating files: 15% (639/4260) Updating files: 16% (682/4260) Updating files: 17% (725/4260) Updating files: 18% (767/4260) Updating files: 19% (810/4260) Updating files: 20% (852/4260) Updating files: 21% (895/4260) Updating files: 22% (938/4260) Updating files: 23% (980/4260) Updating files: 23% (1004/4260) Updating files: 24% (1023/4260) Updating files: 25% (1065/4260) Updating files: 26% (1108/4260) Updating files: 27% (1151/4260) Updating files: 28% (1193/4260) Updating files: 29% (1236/4260) Updating files: 29% (1270/4260) Updating files: 30% (1278/4260) Updating files: 31% (1321/4260) Updating files: 32% (1364/4260) Updating files: 33% (1406/4260) Updating files: 34% (1449/4260) Updating files: 35% (1491/4260) Updating files: 36% (1534/4260) Updating files: 37% (1577/4260) Updating files: 38% (1619/4260) Updating files: 39% (1662/4260) Updating files: 40% (1704/4260) Updating files: 41% (1747/4260) Updating files: 41% (1788/4260) Updating files: 42% (1790/4260) Updating files: 43% (1832/4260) Updating files: 44% (1875/4260) Updating files: 45% (1917/4260) Updating files: 46% (1960/4260) Updating files: 47% (2003/4260) Updating files: 48% (2045/4260) Updating files: 49% (2088/4260) Updating files: 50% (2130/4260) Updating files: 51% (2173/4260) Updating files: 52% (2216/4260) Updating files: 53% (2258/4260) Updating files: 53% (2261/4260) Updating files: 54% (2301/4260) Updating files: 55% (2343/4260) Updating files: 56% (2386/4260) Updating files: 57% (2429/4260) Updating files: 58% (2471/4260) Updating files: 59% (2514/4260) Updating files: 60% (2556/4260) Updating files: 61% (2599/4260) Updating files: 62% (2642/4260) Updating files: 63% (2684/4260) Updating files: 64% (2727/4260) Updating files: 64% (2760/4260) Updating files: 65% (2769/4260) Updating files: 66% (2812/4260) Updating files: 67% (2855/4260) Updating files: 68% (2897/4260) Updating files: 69% (2940/4260) Updating files: 70% (2982/4260) Updating files: 70% (2998/4260) Updating files: 71% (3025/4260) Updating files: 72% (3068/4260) Updating files: 73% (3110/4260) Updating files: 74% (3153/4260) Updating files: 75% (3195/4260) Updating files: 76% (3238/4260) Updating files: 77% (3281/4260) Updating files: 78% (3323/4260) Updating files: 79% (3366/4260) Updating files: 80% (3408/4260) Updating files: 81% (3451/4260) Updating files: 82% (3494/4260) Updating files: 82% (3501/4260) Updating files: 83% (3536/4260) Updating files: 84% (3579/4260) Updating files: 85% (3621/4260) Updating files: 86% (3664/4260) Updating files: 87% (3707/4260) Updating files: 88% (3749/4260) Updating files: 89% (3792/4260) Updating files: 90% (3834/4260) Updating files: 91% (3877/4260) Updating files: 92% (3920/4260) Updating files: 93% (3962/4260) Updating files: 94% (4005/4260) Updating files: 94% (4044/4260) Updating files: 95% (4047/4260) Updating files: 96% (4090/4260) Updating files: 97% (4133/4260) Updating files: 98% (4175/4260) Updating files: 99% (4218/4260) Updating files: 100% (4260/4260) Updating files: 100% (4260/4260), done. 2024-02-26T06:29:22.294 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd xfstests-dev 2024-02-26T06:29:22.296 INFO:tasks.workunit.client.0.smithi006.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-02-26T06:29:22.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:22 smithi090 ceph-mon[35681]: scrub summary: idle 2024-02-26T06:29:22.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:22 smithi090 ceph-mon[35681]: scrub summary: idle 2024-02-26T06:29:22.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:22 smithi006 ceph-mon[31790]: scrub summary: idle 2024-02-26T06:29:22.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:22 smithi006 ceph-mon[31790]: scrub summary: idle 2024-02-26T06:29:22.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:22 smithi167 ceph-mon[36229]: scrub summary: idle 2024-02-26T06:29:22.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:22 smithi167 ceph-mon[36229]: scrub summary: idle 2024-02-26T06:29:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:23 smithi090 ceph-mon[35681]: pgmap v41: 97 pgs: 97 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 756 KiB/s wr, 46 op/s 2024-02-26T06:29:23.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:23 smithi006 ceph-mon[31790]: pgmap v41: 97 pgs: 97 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 756 KiB/s wr, 46 op/s 2024-02-26T06:29:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:23 smithi167 ceph-mon[36229]: pgmap v41: 97 pgs: 97 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 756 KiB/s wr, 46 op/s 2024-02-26T06:29:25.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:25 smithi090 ceph-mon[35681]: pgmap v42: 97 pgs: 97 active+clean; 24 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 483 KiB/s wr, 45 op/s 2024-02-26T06:29:25.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:25 smithi006 ceph-mon[31790]: pgmap v42: 97 pgs: 97 active+clean; 24 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 483 KiB/s wr, 45 op/s 2024-02-26T06:29:25.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:25 smithi167 ceph-mon[36229]: pgmap v42: 97 pgs: 97 active+clean; 24 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 483 KiB/s wr, 45 op/s 2024-02-26T06:29:27.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:27 smithi090 ceph-mon[35681]: pgmap v43: 97 pgs: 97 active+clean; 24 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 418 KiB/s wr, 32 op/s 2024-02-26T06:29:27.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:27 smithi006 ceph-mon[31790]: pgmap v43: 97 pgs: 97 active+clean; 24 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 418 KiB/s wr, 32 op/s 2024-02-26T06:29:27.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:27 smithi167 ceph-mon[36229]: pgmap v43: 97 pgs: 97 active+clean; 24 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 418 KiB/s wr, 32 op/s 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 -- 172.21.15.6:0/1643109736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c103410 msgr2=0x7fa53c1037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 --2- 172.21.15.6:0/1643109736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c103410 0x7fa53c1037f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009900 tx=0x7fa52c02bc40 comp rx=0 tx=0).stop 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 -- 172.21.15.6:0/1643109736 shutdown_connections 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 --2- 172.21.15.6:0/1643109736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1084e0 0x7fa53c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 --2- 172.21.15.6:0/1643109736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103dc0 0x7fa53c107e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 --2- 172.21.15.6:0/1643109736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c103410 0x7fa53c1037f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.514+0000 7fa53be00640 1 -- 172.21.15.6:0/1643109736 >> 172.21.15.6:0/1643109736 conn(0x7fa53c0fec70 msgr2=0x7fa53c101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.515+0000 7fa53be00640 1 -- 172.21.15.6:0/1643109736 shutdown_connections 2024-02-26T06:29:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.515+0000 7fa53be00640 1 -- 172.21.15.6:0/1643109736 wait complete. 2024-02-26T06:29:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.516+0000 7fa53be00640 1 Processor -- start 2024-02-26T06:29:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.516+0000 7fa53be00640 1 -- start start 2024-02-26T06:29:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 0x7fa53c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c1084e0 0x7fa53c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1996c0 0x7fa53c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53be00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c19a190 con 0x7fa53c103410 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c19a300 con 0x7fa53c1996c0 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c19a4c0 con 0x7fa53c1084e0 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 0x7fa53c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 0x7fa53c19f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47592/0 (socket says 172.21.15.6:47592) 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53aa00640 1 -- 172.21.15.6:0/344493840 learned_addr learned my addr 172.21.15.6:0/344493840 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53a000640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c1084e0 0x7fa53c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.517+0000 7fa53b400640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1996c0 0x7fa53c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53a000640 1 -- 172.21.15.6:0/344493840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1996c0 msgr2=0x7fa53c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53a000640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1996c0 0x7fa53c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53a000640 1 -- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 msgr2=0x7fa53c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53a000640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 0x7fa53c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53a000640 1 -- 172.21.15.6:0/344493840 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c009560 con 0x7fa53c1084e0 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53aa00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 0x7fa53c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53b400640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1996c0 0x7fa53c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.518+0000 7fa53a000640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c1084e0 0x7fa53c19faf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa52800d6a0 tx=0x7fa52800db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.519+0000 7fa533400640 1 -- 172.21.15.6:0/344493840 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528005220 con 0x7fa53c1084e0 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.519+0000 7fa533400640 1 -- 172.21.15.6:0/344493840 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa528009d00 con 0x7fa53c1084e0 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.519+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c1a6410 con 0x7fa53c1084e0 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.519+0000 7fa533400640 1 -- 172.21.15.6:0/344493840 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528004dd0 con 0x7fa53c1084e0 2024-02-26T06:29:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.519+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c1a6940 con 0x7fa53c1084e0 2024-02-26T06:29:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.522+0000 7fa533400640 1 -- 172.21.15.6:0/344493840 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fa528009820 con 0x7fa53c1084e0 2024-02-26T06:29:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa54457d740 1 Processor -- start 2024-02-26T06:29:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa533400640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fa5200a23b0 0x7fa5200a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa54457d740 1 -- start start 2024-02-26T06:29:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa533400640 1 -- 172.21.15.6:0/344493840 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fa5280cdbe0 con 0x7fa53c1084e0 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa53b400640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fa5200a23b0 0x7fa5200a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa54457d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.524+0000 7fa54457d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa54457d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa54457d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99975f0 con 0x5616d99dcd30 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa54457d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d9949890 con 0x5616d99e5540 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa54457d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d9946dc0 con 0x5616d99dbfc0 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99dba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47606/0 (socket says 172.21.15.6:47606) 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53aa00640 1 -- 172.21.15.6:0/1888214902 learned_addr learned my addr 172.21.15.6:0/1888214902 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53b400640 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53a000640 1 -- 172.21.15.6:0/1888214902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 msgr2=0x5616d99e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53a000640 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53a000640 1 -- 172.21.15.6:0/1888214902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 msgr2=0x5616d99dba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53a000640 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99dba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.525+0000 7fa53a000640 1 -- 172.21.15.6:0/1888214902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616d98c3ad0 con 0x5616d99dbfc0 2024-02-26T06:29:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.526+0000 7fa53b400640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fa5200a23b0 0x7fa5200a4870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa534009fd0 tx=0x7fa534011b20 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.526+0000 7fa53a000640 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99dc440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5280cb130 tx=0x7fa528099fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866836f42e4113f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.526+0000 7fa527400640 1 -- 172.21.15.6:0/1888214902 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52809e450 con 0x5616d99dbfc0 2024-02-26T06:29:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.526+0000 7fa527400640 1 -- 172.21.15.6:0/1888214902 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5280a2aa0 con 0x5616d99dbfc0 2024-02-26T06:29:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.526+0000 7fa527400640 1 -- 172.21.15.6:0/1888214902 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52809e280 con 0x5616d99dbfc0 2024-02-26T06:29:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 -- 172.21.15.6:0/1888214902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 msgr2=0x5616d99dc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99dc440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5280cb130 tx=0x7fa528099fb0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 -- 172.21.15.6:0/1888214902 shutdown_connections 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99dc440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 --2- 172.21.15.6:0/1888214902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99dba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 -- 172.21.15.6:0/1888214902 >> 172.21.15.6:0/1888214902 conn(0x5616d99a3480 msgr2=0x5616d99d6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 -- 172.21.15.6:0/1888214902 shutdown_connections 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.527+0000 7fa54457d740 1 -- 172.21.15.6:0/1888214902 wait complete. 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.528+0000 7fa54457d740 1 -- start start 2024-02-26T06:29:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.528+0000 7fa54457d740 1 Processor -- start 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99dbfc0 0x5616d99e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 0x5616d99e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99e5540 0x5616d99e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99425a0 con 0x5616d99e5540 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d98e7b20 con 0x5616d99dbfc0 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d98e77c0 con 0x5616d99dcd30 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa54457d740 10 client.0 _ll_register_callbacks cb 0x5616d98fb520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa53a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 0x5616d99e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa53a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 0x5616d99e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36140/0 (socket says 172.21.15.6:36140) 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa53a000640 1 -- 172.21.15.6:0/2335419706 learned_addr learned my addr 172.21.15.6:0/2335419706 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa53aa00640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99dbfc0 0x5616d99e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.529+0000 7fa53b400640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99e5540 0x5616d99e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa53a000640 1 -- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99dbfc0 msgr2=0x5616d99e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa53a000640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99dbfc0 0x5616d99e25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa53a000640 1 -- 172.21.15.6:0/2335419706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99e5540 msgr2=0x5616d99e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa53a000640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99e5540 0x5616d99e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa53a000640 1 -- 172.21.15.6:0/2335419706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c1a6940 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa53a000640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 0x5616d99e2af0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa52809a0b0 tx=0x7fa528005860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3974b2fed354f8f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa51be00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa51be00640 1 -- 172.21.15.6:0/2335419706 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52809e450 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa51be00640 1 -- 172.21.15.6:0/2335419706 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fa528096a20 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.530+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616d9914ca0 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa51be00640 1 -- 172.21.15.6:0/2335419706 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa528096be0 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 10 client.25915 fetch_fsmap learned FSMap version 34 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5616d9940c20 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa51be00640 1 -- 172.21.15.6:0/2335419706 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616d9914ca0 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa51be00640 1 -- 172.21.15.6:0/2335419706 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa528099270 con 0x5616d99dcd30 2024-02-26T06:29:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 10 client.25915 fetch_fsmap finished waiting for FSMap version 34 2024-02-26T06:29:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 20 client.25915 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d99f51d0 0x5616d99f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 4 client.25915 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.531+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616d9938cb0 con 0x5616d99f51d0 2024-02-26T06:29:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.532+0000 7fa53aa00640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d99f51d0 0x5616d99f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.532+0000 7fa53aa00640 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d99f51d0 0x5616d99f75f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2dc315bc0ebd1c9 server_cookie=fe73c40fcaceb3c4 in_seq=0 out_seq=0 2024-02-26T06:29:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.532+0000 7fa51be00640 10 client.25915 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa51be00640 1 -- 172.21.15.6:0/2335419706 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616d9938cb0 con 0x5616d99f51d0 2024-02-26T06:29:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa51be00640 10 client.25915 handle_command_reply: tid=0 2024-02-26T06:29:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 client.25915 shutdown 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 msgr2=0x5616d99e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 0x5616d99e2af0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa52809a0b0 tx=0x7fa528005860 comp rx=0 tx=0).stop 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 shutdown_connections 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d99f51d0 0x5616d99f75f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99e5540 0x5616d99e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dcd30 0x5616d99e2af0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 --2- 172.21.15.6:0/2335419706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99dbfc0 0x5616d99e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 >> 172.21.15.6:0/2335419706 conn(0x5616d99a3480 msgr2=0x5616d99d8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa51be00640 0 client.25915 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 shutdown_connections 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.533+0000 7fa54457d740 1 -- 172.21.15.6:0/2335419706 wait complete. 2024-02-26T06:29:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.534+0000 7fa54457d740 20 client.25915 trim_cache size 0 max 16384 2024-02-26T06:29:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 Processor -- start 2024-02-26T06:29:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 -- start start 2024-02-26T06:29:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d968ae60 con 0x5616d99dcd30 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99d6560 con 0x5616d99e5540 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa54457d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99d7ee0 con 0x5616d99dbfc0 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99e4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36148/0 (socket says 172.21.15.6:36148) 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa53a000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.556+0000 7fa53aa00640 1 -- 172.21.15.6:0/313411941 learned_addr learned my addr 172.21.15.6:0/313411941 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53b400640 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53aa00640 1 -- 172.21.15.6:0/313411941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 msgr2=0x5616d99e0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53aa00640 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53aa00640 1 -- 172.21.15.6:0/313411941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 msgr2=0x5616d99fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53aa00640 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53aa00640 1 -- 172.21.15.6:0/313411941 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616d98dd500 con 0x5616d99dbfc0 2024-02-26T06:29:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53b400640 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa53aa00640 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99e4ce0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa52c02c250 tx=0x7fa52c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34057ea7d9674d39 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa526a00640 1 -- 172.21.15.6:0/313411941 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c045040 con 0x5616d99dbfc0 2024-02-26T06:29:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa526a00640 1 -- 172.21.15.6:0/313411941 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa52c047040 con 0x5616d99dbfc0 2024-02-26T06:29:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.557+0000 7fa526a00640 1 -- 172.21.15.6:0/313411941 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004df0 con 0x5616d99dbfc0 2024-02-26T06:29:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 -- 172.21.15.6:0/313411941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 msgr2=0x5616d99e4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99e4ce0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa52c02c250 tx=0x7fa52c005a20 comp rx=0 tx=0).stop 2024-02-26T06:29:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 -- 172.21.15.6:0/313411941 shutdown_connections 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 --2- 172.21.15.6:0/313411941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99e4ce0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 -- 172.21.15.6:0/313411941 >> 172.21.15.6:0/313411941 conn(0x5616d99a3480 msgr2=0x5616d99dd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 -- 172.21.15.6:0/313411941 shutdown_connections 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.558+0000 7fa54457d740 1 -- 172.21.15.6:0/313411941 wait complete. 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.559+0000 7fa54457d740 1 -- start start 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.559+0000 7fa54457d740 1 Processor -- start 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.559+0000 7fa54457d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99ec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.559+0000 7fa54457d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.559+0000 7fa54457d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.559+0000 7fa54457d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99eabe0 con 0x5616d99dcd30 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa54457d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99ead50 con 0x5616d99e5540 2024-02-26T06:29:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa54457d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616d99eaf10 con 0x5616d99dbfc0 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa54457d740 10 client.0 _ll_register_callbacks cb 0x7fa52c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99ec030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36158/0 (socket says 172.21.15.6:36158) 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 -- 172.21.15.6:0/4087671620 learned_addr learned my addr 172.21.15.6:0/4087671620 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53a000640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53b400640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 -- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 msgr2=0x5616d99ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99ea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 -- 172.21.15.6:0/4087671620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 msgr2=0x5616d99ec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99ec570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 -- 172.21.15.6:0/4087671620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c1108c0 con 0x5616d99dbfc0 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53b400640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53a000640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99ec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa53aa00640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99ec030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5180dae6ff44d5f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.560+0000 7fa51be00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa51be00640 1 -- 172.21.15.6:0/4087671620 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c045040 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa51be00640 1 -- 172.21.15.6:0/4087671620 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fa52c04c710 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616d9967d00 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa51be00640 1 -- 172.21.15.6:0/4087671620 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa52c04c8d0 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa54457d740 10 client.25921 fetch_fsmap learned FSMap version 34 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5616d9903290 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa51be00640 1 -- 172.21.15.6:0/4087671620 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616d9967d00 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.561+0000 7fa51be00640 1 -- 172.21.15.6:0/4087671620 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa52c03c040 con 0x5616d99dbfc0 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa54457d740 10 client.25921 fetch_fsmap finished waiting for FSMap version 34 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa54457d740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa54457d740 20 client.25921 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa54457d740 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d9a4c720 0x5616d9a4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa54457d740 4 client.25921 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616d9642600 con 0x5616d9a4c720 2024-02-26T06:29:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa53a000640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d9a4c720 0x5616d9a4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa53a000640 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d9a4c720 0x5616d9a4eb40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c99f4dc4cce5eb3 server_cookie=f3821c60f825a2f4 in_seq=0 out_seq=0 2024-02-26T06:29:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.562+0000 7fa51be00640 10 client.25921 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa51be00640 1 -- 172.21.15.6:0/4087671620 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616d9642600 con 0x5616d9a4c720 2024-02-26T06:29:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa51be00640 10 client.25921 handle_command_reply: tid=0 2024-02-26T06:29:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 client.25921 shutdown 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 msgr2=0x5616d99ec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99ec030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c005850 comp rx=0 tx=0).stop 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 shutdown_connections 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x5616d9a4c720 0x5616d9a4eb40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616d99e5540 0x5616d99ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5616d99dcd30 0x5616d99ec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 --2- 172.21.15.6:0/4087671620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5616d99dbfc0 0x5616d99ec030 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa51be00640 0 client.25921 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 >> 172.21.15.6:0/4087671620 conn(0x5616d99a3480 msgr2=0x5616d99fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 shutdown_connections 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.571+0000 7fa54457d740 1 -- 172.21.15.6:0/4087671620 wait complete. 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa54457d740 20 client.25921 trim_cache size 0 max 16384 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:29:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fa5200a23b0 msgr2=0x7fa5200a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fa5200a23b0 0x7fa5200a4870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa534009fd0 tx=0x7fa534011b20 comp rx=0 tx=0).stop 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c1084e0 msgr2=0x7fa53c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c1084e0 0x7fa53c19faf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa52800d6a0 tx=0x7fa52800db60 comp rx=0 tx=0).stop 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 shutdown_connections 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fa5200a23b0 0x7fa5200a4870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c1996c0 0x7fa53c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c1084e0 0x7fa53c19faf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 --2- 172.21.15.6:0/344493840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa53c103410 0x7fa53c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.572+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 >> 172.21.15.6:0/344493840 conn(0x7fa53c0fec70 msgr2=0x7fa53c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.573+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 shutdown_connections 2024-02-26T06:29:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:28.573+0000 7fa53be00640 1 -- 172.21.15.6:0/344493840 wait complete. 2024-02-26T06:29:29.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:29 smithi090 ceph-mon[35681]: pgmap v44: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-02-26T06:29:29.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:29 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:29.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:29 smithi006 ceph-mon[31790]: pgmap v44: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-02-26T06:29:29.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:29 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:29 smithi167 ceph-mon[36229]: pgmap v44: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-02-26T06:29:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:29 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:30.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:30 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:30.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:30 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:30.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:30 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:29:31.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:31 smithi090 ceph-mon[35681]: pgmap v45: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-02-26T06:29:31.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:31 smithi006 ceph-mon[31790]: pgmap v45: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-02-26T06:29:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:31 smithi167 ceph-mon[36229]: pgmap v45: 97 pgs: 97 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-02-26T06:29:33.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T06:29:33.512 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T06:29:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:33 smithi090 ceph-mon[35681]: pgmap v46: 97 pgs: 97 active+clean; 49 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 494 op/s 2024-02-26T06:29:33.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:33 smithi006 ceph-mon[31790]: pgmap v46: 97 pgs: 97 active+clean; 49 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 494 op/s 2024-02-26T06:29:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:33 smithi167 ceph-mon[36229]: pgmap v46: 97 pgs: 97 active+clean; 49 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 494 op/s 2024-02-26T06:29:33.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 -- 172.21.15.6:0/2186877562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 msgr2=0x7f56a4114710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 --2- 172.21.15.6:0/2186877562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a4114710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f569c009d00 tx=0x7f569c02c0f0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 -- 172.21.15.6:0/2186877562 shutdown_connections 2024-02-26T06:29:33.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 --2- 172.21.15.6:0/2186877562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a4114710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 --2- 172.21.15.6:0/2186877562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a41078a0 0x7f56a4107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 --2- 172.21.15.6:0/2186877562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56a4109590 0x7f56a4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 -- 172.21.15.6:0/2186877562 >> 172.21.15.6:0/2186877562 conn(0x7f56a4077020 msgr2=0x7f56a4105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:33.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 -- 172.21.15.6:0/2186877562 shutdown_connections 2024-02-26T06:29:33.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.749+0000 7f56ab000640 1 -- 172.21.15.6:0/2186877562 wait complete. 2024-02-26T06:29:33.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.750+0000 7f56ab000640 1 Processor -- start 2024-02-26T06:29:33.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.750+0000 7f56ab000640 1 -- start start 2024-02-26T06:29:33.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.750+0000 7f56ab000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a41078a0 0x7f56a419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56ab000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56ab000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56a4109590 0x7f56a41a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56ab000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a2d30 con 0x7f56a4109590 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56ab000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a2ea0 con 0x7f56a41078a0 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56ab000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a3060 con 0x7f56a4108240 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a9000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56a4109590 0x7f56a41a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a41078a0 0x7f56a419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36182/0 (socket says 172.21.15.6:36182) 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 -- 172.21.15.6:0/1078077422 learned_addr learned my addr 172.21.15.6:0/1078077422 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 -- 172.21.15.6:0/1078077422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a41078a0 msgr2=0x7f56a419dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a41078a0 0x7f56a419dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 -- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56a4109590 msgr2=0x7f56a41a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56a4109590 0x7f56a41a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.751+0000 7f56a3400640 1 -- 172.21.15.6:0/1078077422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009900 con 0x7f56a4108240 2024-02-26T06:29:33.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.752+0000 7f56a3400640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a419e400 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9f0 tx=0x7f569000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:33.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.752+0000 7f56a0c00640 1 -- 172.21.15.6:0/1078077422 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690009720 con 0x7f56a4108240 2024-02-26T06:29:33.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.752+0000 7f56a0c00640 1 -- 172.21.15.6:0/1078077422 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f569000cd80 con 0x7f56a4108240 2024-02-26T06:29:33.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.752+0000 7f56a0c00640 1 -- 172.21.15.6:0/1078077422 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690010690 con 0x7f56a4108240 2024-02-26T06:29:33.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.752+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a41a3340 con 0x7f56a4108240 2024-02-26T06:29:33.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.752+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a41aabc0 con 0x7f56a4108240 2024-02-26T06:29:33.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.755+0000 7f56adf76740 1 Processor -- start 2024-02-26T06:29:33.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.755+0000 7f56adf76740 1 -- start start 2024-02-26T06:29:33.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56adf76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56adf76740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56adf76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56adf76740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f141120 con 0x558f5f1d2290 2024-02-26T06:29:33.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56adf76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f13e650 con 0x558f5f1d2f70 2024-02-26T06:29:33.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56adf76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f139e30 con 0x558f5f1db600 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56a9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56a9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56558/0 (socket says 172.21.15.6:56558) 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56a9000640 1 -- 172.21.15.6:0/4093392044 learned_addr learned my addr 172.21.15.6:0/4093392044 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.756+0000 7f56a0c00640 1 -- 172.21.15.6:0/1078077422 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f56900046c0 con 0x7f56a4108240 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a3400640 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a9000640 1 -- 172.21.15.6:0/4093392044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 msgr2=0x558f5f1dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a9000640 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a9000640 1 -- 172.21.15.6:0/4093392044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 msgr2=0x558f5f1d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a9000640 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a9000640 1 -- 172.21.15.6:0/4093392044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5f0f3030 con 0x558f5f1d2f70 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a3400640 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:33.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f56a9000640 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d1cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f569c004af0 tx=0x7f569c032d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b043197f4cd13e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:33.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.757+0000 7f5699600640 1 -- 172.21.15.6:0/4093392044 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c034440 con 0x558f5f1d2f70 2024-02-26T06:29:33.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f5699600640 1 -- 172.21.15.6:0/4093392044 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f569c0345a0 con 0x558f5f1d2f70 2024-02-26T06:29:33.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f5699600640 1 -- 172.21.15.6:0/4093392044 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c003b70 con 0x558f5f1d2f70 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56a0c00640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f56880a2350 0x7f56880a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56a0c00640 1 -- 172.21.15.6:0/1078077422 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5690014070 con 0x7f56a4108240 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56a3e00640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f56880a2350 0x7f56880a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 -- 172.21.15.6:0/4093392044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 msgr2=0x558f5f1d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d1cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f569c004af0 tx=0x7f569c032d70 comp rx=0 tx=0).stop 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 -- 172.21.15.6:0/4093392044 shutdown_connections 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 --2- 172.21.15.6:0/4093392044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d1cc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 -- 172.21.15.6:0/4093392044 >> 172.21.15.6:0/4093392044 conn(0x558f5f199570 msgr2=0x558f5f1ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.758+0000 7f56adf76740 1 -- 172.21.15.6:0/4093392044 shutdown_connections 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.759+0000 7f56adf76740 1 -- 172.21.15.6:0/4093392044 wait complete. 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.759+0000 7f56a3e00640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f56880a2350 0x7f56880a4810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x558f5f1d34c0 tx=0x7f569400b410 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.759+0000 7f56adf76740 1 -- start start 2024-02-26T06:29:33.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.759+0000 7f56adf76740 1 Processor -- start 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f0dee90 con 0x558f5f1d2290 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f0deb30 con 0x558f5f1d2f70 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5eea81c0 con 0x558f5f1db600 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56adf76740 10 client.0 _ll_register_callbacks cb 0x558f5f0f2930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56a9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56568/0 (socket says 172.21.15.6:56568) 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56a9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1d8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36192/0 (socket says 172.21.15.6:36192) 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.760+0000 7f56a3400640 1 -- 172.21.15.6:0/2550180800 learned_addr learned my addr 172.21.15.6:0/2550180800 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a3400640 1 -- 172.21.15.6:0/2550180800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 msgr2=0x558f5f1d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a3400640 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a3400640 1 -- 172.21.15.6:0/2550180800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 msgr2=0x558f5f1d8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a3400640 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a3400640 1 -- 172.21.15.6:0/2550180800 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a4114fa0 con 0x558f5f1d2f70 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a9000640 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56a3400640 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8a70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f569009ab70 tx=0x7f569009b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=134cf435f5368f0f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f568e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f568e000640 1 -- 172.21.15.6:0/2550180800 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569009efc0 con 0x558f5f1d2f70 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f568e000640 1 -- 172.21.15.6:0/2550180800 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f56900a3610 con 0x558f5f1d2f70 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f5ee397e0 con 0x558f5f1d2f70 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f568e000640 1 -- 172.21.15.6:0/2550180800 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56900b9330 con 0x558f5f1d2f70 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56adf76740 10 client.25949 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.761+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558f5f1384b0 con 0x558f5f1d2f70 2024-02-26T06:29:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f568e000640 1 -- 172.21.15.6:0/2550180800 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558f5ee397e0 con 0x558f5f1d2f70 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f568e000640 1 -- 172.21.15.6:0/2550180800 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f569001f060 con 0x558f5f1d2f70 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56adf76740 10 client.25949 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56adf76740 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56adf76740 20 client.25949 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56adf76740 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f1eb710 0x558f5f1edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56adf76740 4 client.25949 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f5ee76720 con 0x558f5f1eb710 2024-02-26T06:29:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.762+0000 7f56a3e00640 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f1eb710 0x558f5f1edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.763+0000 7f56a3e00640 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f1eb710 0x558f5f1edb30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d4b695b4ef038c server_cookie=15b9e58de8345cfc in_seq=0 out_seq=0 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.763+0000 7f568e000640 10 client.25949 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f568e000640 1 -- 172.21.15.6:0/2550180800 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f5ee76720 con 0x558f5f1eb710 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f568e000640 10 client.25949 handle_command_reply: tid=0 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 client.25949 shutdown 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 msgr2=0x558f5f1d8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8a70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f569009ab70 tx=0x7f569009b570 comp rx=0 tx=0).stop 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 shutdown_connections 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f1eb710 0x558f5f1edb30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1db600 0x558f5f1d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f568e000640 0 client.25949 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8a70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 --2- 172.21.15.6:0/2550180800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1d2290 0x558f5f1d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 >> 172.21.15.6:0/2550180800 conn(0x558f5f199570 msgr2=0x558f5f1cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 shutdown_connections 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 1 -- 172.21.15.6:0/2550180800 wait complete. 2024-02-26T06:29:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.764+0000 7f56adf76740 20 client.25949 trim_cache size 0 max 16384 2024-02-26T06:29:33.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.781+0000 7f56adf76740 1 Processor -- start 2024-02-26T06:29:33.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.781+0000 7f56adf76740 1 -- start start 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56adf76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56adf76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56adf76740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1e2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56adf76740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f1c85b0 con 0x558f5f1db600 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56adf76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f1d1b00 con 0x558f5f1d2f70 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56adf76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f1d1560 con 0x558f5f1d2290 2024-02-26T06:29:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1dac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36200/0 (socket says 172.21.15.6:36200) 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56a3e00640 1 -- 172.21.15.6:0/1659514432 learned_addr learned my addr 172.21.15.6:0/1659514432 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.782+0000 7f56a9000640 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1e2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3400640 1 -- 172.21.15.6:0/1659514432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 msgr2=0x558f5f1dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3400640 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1dac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3400640 1 -- 172.21.15.6:0/1659514432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 msgr2=0x558f5f1e2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3400640 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1e2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3400640 1 -- 172.21.15.6:0/1659514432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5f0d4920 con 0x558f5f1d2f70 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3e00640 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f56a3400640 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f56900cc420 tx=0x7f569009a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f961195cc622f98 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f5698c00640 1 -- 172.21.15.6:0/1659514432 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569009a9b0 con 0x558f5f1d2f70 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f5698c00640 1 -- 172.21.15.6:0/1659514432 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56900a3610 con 0x558f5f1d2f70 2024-02-26T06:29:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.783+0000 7f5698c00640 1 -- 172.21.15.6:0/1659514432 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56900a33e0 con 0x558f5f1d2f70 2024-02-26T06:29:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 -- 172.21.15.6:0/1659514432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 msgr2=0x558f5f1d8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f56900cc420 tx=0x7f569009a720 comp rx=0 tx=0).stop 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 -- 172.21.15.6:0/1659514432 shutdown_connections 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1e2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1d8dd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 --2- 172.21.15.6:0/1659514432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 -- 172.21.15.6:0/1659514432 >> 172.21.15.6:0/1659514432 conn(0x558f5f199570 msgr2=0x558f5f1cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 -- 172.21.15.6:0/1659514432 shutdown_connections 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.784+0000 7f56adf76740 1 -- 172.21.15.6:0/1659514432 wait complete. 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 -- start start 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 Processor -- start 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f1df580 con 0x558f5f1db600 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f1df6f0 con 0x558f5f1d2f70 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5f1df8b0 con 0x558f5f1d2290 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.785+0000 7f56adf76740 10 client.0 _ll_register_callbacks cb 0x7f5690004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1e26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36206/0 (socket says 172.21.15.6:36206) 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 -- 172.21.15.6:0/2793105182 learned_addr learned my addr 172.21.15.6:0/2793105182 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3400640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a9000640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 -- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 msgr2=0x558f5f1deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 -- 172.21.15.6:0/2793105182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 msgr2=0x558f5f1df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 -- 172.21.15.6:0/2793105182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569401e660 con 0x558f5f1d2290 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a9000640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f56a3e00640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1e26c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5694029400 tx=0x7f56940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301704332dce045e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.786+0000 7f568e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f568e000640 1 -- 172.21.15.6:0/2793105182 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694024040 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f569c0345a0 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f568e000640 1 -- 172.21.15.6:0/2793105182 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f569401e970 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f568e000640 1 -- 172.21.15.6:0/2793105182 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5694016070 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 10 client.25936 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558f5f1384b0 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f568e000640 1 -- 172.21.15.6:0/2793105182 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f569c0345a0 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f568e000640 1 -- 172.21.15.6:0/2793105182 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5694010430 con 0x558f5f1d2290 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 10 client.25936 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 20 client.25936 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f243360 0x558f5f245780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 4 client.25936 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T06:29:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.787+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f5eec5890 con 0x558f5f243360 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.788+0000 7f56a3400640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f243360 0x558f5f245780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56a3400640 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f243360 0x558f5f245780 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533645a7535634df server_cookie=aa176d0d2795e86b in_seq=0 out_seq=0 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f568e000640 10 client.25936 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f568e000640 1 -- 172.21.15.6:0/2793105182 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558f5eec5890 con 0x558f5f243360 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f568e000640 10 client.25936 handle_command_reply: tid=0 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 client.25936 shutdown 2024-02-26T06:29:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 msgr2=0x558f5f1e26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1e26c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5694029400 tx=0x7f56940046c0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 shutdown_connections 2024-02-26T06:29:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x558f5f243360 0x558f5f245780 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558f5f1db600 0x558f5f1df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f5f1d2f70 0x558f5f1deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f568e000640 0 client.25936 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 --2- 172.21.15.6:0/2793105182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f5f1d2290 0x558f5f1e26c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 >> 172.21.15.6:0/2793105182 conn(0x558f5f199570 msgr2=0x558f5f1d6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 shutdown_connections 2024-02-26T06:29:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 1 -- 172.21.15.6:0/2793105182 wait complete. 2024-02-26T06:29:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.789+0000 7f56adf76740 20 client.25936 trim_cache size 0 max 16384 2024-02-26T06:29:33.795 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:29:33.795 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2024-02-26T06:29:33.795 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2024-02-26T06:29:33.795 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:29:33.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f56880a2350 msgr2=0x7f56880a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f56880a2350 0x7f56880a4810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x558f5f1d34c0 tx=0x7f569400b410 comp rx=0 tx=0).stop 2024-02-26T06:29:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 msgr2=0x7f56a419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a419e400 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9f0 tx=0x7f569000edf0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 shutdown_connections 2024-02-26T06:29:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f56880a2350 0x7f56880a4810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56a4109590 0x7f56a41a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56a4108240 0x7f56a419e400 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 --2- 172.21.15.6:0/1078077422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a41078a0 0x7f56a419dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 >> 172.21.15.6:0/1078077422 conn(0x7f56a4077020 msgr2=0x7f56a41061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 shutdown_connections 2024-02-26T06:29:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:33.790+0000 7f56ab000640 1 -- 172.21.15.6:0/1078077422 wait complete. 2024-02-26T06:29:33.809 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T06:29:33.810 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 -- 172.21.15.6:0/2906780673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41009f0 msgr2=0x7f69e4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 --2- 172.21.15.6:0/2906780673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41009f0 0x7f69e4104eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009900 tx=0x7f69cc02bcc0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 -- 172.21.15.6:0/2906780673 shutdown_connections 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 --2- 172.21.15.6:0/2906780673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41054d0 0x7f69e4105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 --2- 172.21.15.6:0/2906780673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41009f0 0x7f69e4104eb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 --2- 172.21.15.6:0/2906780673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41000d0 0x7f69e41004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 -- 172.21.15.6:0/2906780673 >> 172.21.15.6:0/2906780673 conn(0x7f69e40fbb00 msgr2=0x7f69e40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 -- 172.21.15.6:0/2906780673 shutdown_connections 2024-02-26T06:29:34.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.035+0000 7f69eb400640 1 -- 172.21.15.6:0/2906780673 wait complete. 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.036+0000 7f69eb400640 1 Processor -- start 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 -- start start 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 0x7f69e419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41009f0 0x7f69e419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41054d0 0x7f69e41a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41a2c30 con 0x7f69e41000d0 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41a2da0 con 0x7f69e41009f0 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69eb400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41a2f60 con 0x7f69e41054d0 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69e8a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 0x7f69e419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69e9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41054d0 0x7f69e41a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69e8a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 0x7f69e419dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47694/0 (socket says 172.21.15.6:47694) 2024-02-26T06:29:34.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69e8a00640 1 -- 172.21.15.6:0/3518395747 learned_addr learned my addr 172.21.15.6:0/3518395747 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.037+0000 7f69dbe00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41009f0 0x7f69e419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e8a00640 1 -- 172.21.15.6:0/3518395747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41054d0 msgr2=0x7f69e41a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e8a00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41054d0 0x7f69e41a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e8a00640 1 -- 172.21.15.6:0/3518395747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41009f0 msgr2=0x7f69e419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e8a00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41009f0 0x7f69e419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e8a00640 1 -- 172.21.15.6:0/3518395747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4009490 con 0x7f69e41000d0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69dbe00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41009f0 0x7f69e419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e8a00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 0x7f69e419dd70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f69d400ea60 tx=0x7f69d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.038+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d400cde0 con 0x7f69e41000d0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.039+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69d40035f0 con 0x7f69e41000d0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.039+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc009560 con 0x7f69e41000d0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.039+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4010690 con 0x7f69e41000d0 2024-02-26T06:29:34.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.039+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e41aa8e0 con 0x7f69e41000d0 2024-02-26T06:29:34.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.040+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e406a140 con 0x7f69e41000d0 2024-02-26T06:29:34.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.043+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f69d4003af0 con 0x7f69e41000d0 2024-02-26T06:29:34.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.044+0000 7f69e2000640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f69c80a2350 0x7f69c80a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.044+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f69d4014070 con 0x7f69e41000d0 2024-02-26T06:29:34.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.044+0000 7f69dbe00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f69c80a2350 0x7f69c80a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.045+0000 7f69dbe00640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f69c80a2350 0x7f69c80a4810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c00 tx=0x7f69cc005a20 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.046+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69d4097400 con 0x7f69e41000d0 2024-02-26T06:29:34.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.330+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69e4068450 con 0x7f69e41000d0 2024-02-26T06:29:34.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.332+0000 7f69e2000640 1 -- 172.21.15.6:0/3518395747 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11371 (secure 0 0 0) 0x7f69d409aac0 con 0x7f69e41000d0 2024-02-26T06:29:34.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:29:34.333 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:29:29.309992+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":64,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:29:34.333 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 35 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f69c80a2350 msgr2=0x7f69c80a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f69c80a2350 0x7f69c80a4810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c00 tx=0x7f69cc005a20 comp rx=0 tx=0).stop 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 msgr2=0x7f69e419dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 0x7f69e419dd70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f69d400ea60 tx=0x7f69d400ef20 comp rx=0 tx=0).stop 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 shutdown_connections 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f69c80a2350 0x7f69c80a4810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e41054d0 0x7f69e41a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69e41009f0 0x7f69e419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 --2- 172.21.15.6:0/3518395747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69e41000d0 0x7f69e419dd70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 >> 172.21.15.6:0/3518395747 conn(0x7f69e40fbb00 msgr2=0x7f69e40fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 shutdown_connections 2024-02-26T06:29:34.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.335+0000 7f69eb400640 1 -- 172.21.15.6:0/3518395747 wait complete. 2024-02-26T06:29:34.353 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:34.354 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-26T06:29:34.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 -- 172.21.15.6:0/3826195710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04100c40 msgr2=0x7fdf04104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 --2- 172.21.15.6:0/3826195710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04100c40 0x7fdf04104c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009900 tx=0x7fdeec02bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 -- 172.21.15.6:0/3826195710 shutdown_connections 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 --2- 172.21.15.6:0/3826195710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04105360 0x7fdf0410cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 --2- 172.21.15.6:0/3826195710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04100c40 0x7fdf04104c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 --2- 172.21.15.6:0/3826195710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 0x7fdf04100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 -- 172.21.15.6:0/3826195710 >> 172.21.15.6:0/3826195710 conn(0x7fdf04077230 msgr2=0x7fdf04077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 -- 172.21.15.6:0/3826195710 shutdown_connections 2024-02-26T06:29:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.600+0000 7fdf09e00640 1 -- 172.21.15.6:0/3826195710 wait complete. 2024-02-26T06:29:34.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.604+0000 7fdf09e00640 1 Processor -- start 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.604+0000 7fdf09e00640 1 -- start start 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.604+0000 7fdf09e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 0x7fdf04109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.605+0000 7fdf09e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 0x7fdf04109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.605+0000 7fdf09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04109cc0 0x7fdf041a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.605+0000 7fdf09e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041aa390 con 0x7fdf04100290 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.605+0000 7fdf09e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041aa4d0 con 0x7fdf04109cc0 2024-02-26T06:29:34.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.605+0000 7fdf09e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041aa640 con 0x7fdf04105360 2024-02-26T06:29:34.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.606+0000 7fdefa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 0x7fdf04109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.606+0000 7fdefa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 0x7fdf04109780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36228/0 (socket says 172.21.15.6:36228) 2024-02-26T06:29:34.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.606+0000 7fdefa000640 1 -- 172.21.15.6:0/361937017 learned_addr learned my addr 172.21.15.6:0/361937017 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.606+0000 7fdf03400640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04109cc0 0x7fdf041a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.606+0000 7fdf02a00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 0x7fdf04109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.607+0000 7fdf03400640 1 -- 172.21.15.6:0/361937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 msgr2=0x7fdf04109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.607+0000 7fdf03400640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 0x7fdf04109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.607+0000 7fdf03400640 1 -- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 msgr2=0x7fdf04109240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.608+0000 7fdf03400640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 0x7fdf04109240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.608+0000 7fdf03400640 1 -- 172.21.15.6:0/361937017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec009560 con 0x7fdf04109cc0 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.608+0000 7fdefa000640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 0x7fdf04109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.608+0000 7fdf02a00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 0x7fdf04109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.608+0000 7fdf03400640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04109cc0 0x7fdf041a9e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d5c0 tx=0x7fdefc00da80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.609+0000 7fdf08a00640 1 -- 172.21.15.6:0/361937017 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc005220 con 0x7fdf04109cc0 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.609+0000 7fdf08a00640 1 -- 172.21.15.6:0/361937017 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdefc004c80 con 0x7fdf04109cc0 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.609+0000 7fdf08a00640 1 -- 172.21.15.6:0/361937017 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc009d50 con 0x7fdf04109cc0 2024-02-26T06:29:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.609+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf041aa920 con 0x7fdf04109cc0 2024-02-26T06:29:34.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.610+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf041aadb0 con 0x7fdf04109cc0 2024-02-26T06:29:34.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.617+0000 7fdf0cdf6740 1 Processor -- start 2024-02-26T06:29:34.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.618+0000 7fdf08a00640 1 -- 172.21.15.6:0/361937017 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fdefc0046c0 con 0x7fdf04109cc0 2024-02-26T06:29:34.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.618+0000 7fdf0cdf6740 1 -- start start 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c70dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c716bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c719500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c67c1e0 con 0x55560c717110 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c679710 con 0x55560c70e530 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c674ef0 con 0x55560c70dc10 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf02a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c716bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf02a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c716bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56624/0 (socket says 172.21.15.6:56624) 2024-02-26T06:29:34.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.619+0000 7fdf02a00640 1 -- 172.21.15.6:0/961418520 learned_addr learned my addr 172.21.15.6:0/961418520 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf02a00640 1 -- 172.21.15.6:0/961418520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 msgr2=0x55560c70dff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf02a00640 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c70dff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf02a00640 1 -- 172.21.15.6:0/961418520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 msgr2=0x55560c719500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf02a00640 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c719500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf02a00640 1 -- 172.21.15.6:0/961418520 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55560c636dc0 con 0x55560c70e530 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf08a00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fdee80a2420 0x7fdee80a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.620+0000 7fdf08a00640 1 -- 172.21.15.6:0/361937017 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fdefc0cdd90 con 0x7fdf04109cc0 2024-02-26T06:29:34.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.621+0000 7fdefa000640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fdee80a2420 0x7fdee80a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.621+0000 7fdf02a00640 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c716bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdef40104a0 tx=0x7fdef4010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0908098fbd75d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.623+0000 7fdef3e00640 1 -- 172.21.15.6:0/961418520 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4022710 con 0x55560c70e530 2024-02-26T06:29:34.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.623+0000 7fdef3e00640 1 -- 172.21.15.6:0/961418520 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdef4026070 con 0x55560c70e530 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.623+0000 7fdef3e00640 1 -- 172.21.15.6:0/961418520 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef400ecb0 con 0x55560c70e530 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdefa000640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fdee80a2420 0x7fdee80a48e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf0410aa90 tx=0x7fdeec005b50 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/961418520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 msgr2=0x55560c716bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c716bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdef40104a0 tx=0x7fdef4010960 comp rx=0 tx=0).stop 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/961418520 shutdown_connections 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c719500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c716bd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/961418520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c70dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/961418520 >> 172.21.15.6:0/961418520 conn(0x55560c708340 msgr2=0x55560c708720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/961418520 shutdown_connections 2024-02-26T06:29:34.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.625+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/961418520 wait complete. 2024-02-26T06:29:34.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.626+0000 7fdf0cdf6740 1 -- start start 2024-02-26T06:29:34.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.626+0000 7fdf0cdf6740 1 Processor -- start 2024-02-26T06:29:34.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c70dc10 0x55560c714210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c714750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c717110 0x55560c714c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c61a120 con 0x55560c70dc10 2024-02-26T06:29:34.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c619dc0 con 0x55560c70e530 2024-02-26T06:29:34.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c3e31c0 con 0x55560c717110 2024-02-26T06:29:34.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.627+0000 7fdf0cdf6740 10 client.0 _ll_register_callbacks cb 0x55560c669280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdefa000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c714750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdefa000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c714750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56634/0 (socket says 172.21.15.6:56634) 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdefa000640 1 -- 172.21.15.6:0/2796144047 learned_addr learned my addr 172.21.15.6:0/2796144047 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf03400640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c717110 0x55560c714c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf02a00640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c70dc10 0x55560c714210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf03400640 1 -- 172.21.15.6:0/2796144047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 msgr2=0x55560c714750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf03400640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c714750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf03400640 1 -- 172.21.15.6:0/2796144047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c70dc10 msgr2=0x55560c714210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf03400640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c70dc10 0x55560c714210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdf03400640 1 -- 172.21.15.6:0/2796144047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef4009d10 con 0x55560c717110 2024-02-26T06:29:34.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.628+0000 7fdefa000640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c714750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:34.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.629+0000 7fdf03400640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c717110 0x55560c714c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0cb340 tx=0x7fdefc09a3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163fade084627cde server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.629+0000 7fdef0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.629+0000 7fdef0c00640 1 -- 172.21.15.6:0/2796144047 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc09a510 con 0x55560c717110 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.629+0000 7fdef0c00640 1 -- 172.21.15.6:0/2796144047 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fdefc09e660 con 0x55560c717110 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.629+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55560c3747e0 con 0x55560c717110 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdef0c00640 1 -- 172.21.15.6:0/2796144047 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc0a2cb0 con 0x55560c717110 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 10 client.25951 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55560c673570 con 0x55560c717110 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdef0c00640 1 -- 172.21.15.6:0/2796144047 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55560c3747e0 con 0x55560c717110 2024-02-26T06:29:34.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdef0c00640 1 -- 172.21.15.6:0/2796144047 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdefc0cb620 con 0x55560c717110 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 10 client.25951 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 10 client.25951 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 20 client.25951 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c727320 0x55560c729740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 4 client.25951 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.630+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55560c3b1720 con 0x55560c727320 2024-02-26T06:29:34.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.631+0000 7fdf02a00640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c727320 0x55560c729740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.632+0000 7fdef0c00640 10 client.25951 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.632+0000 7fdf02a00640 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c727320 0x55560c729740 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=973c27c1ffbe2ac0 server_cookie=837f18020c084901 in_seq=0 out_seq=0 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.632+0000 7fdef0c00640 1 -- 172.21.15.6:0/2796144047 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55560c3b1720 con 0x55560c727320 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.632+0000 7fdef0c00640 10 client.25951 handle_command_reply: tid=0 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.632+0000 7fdf0cdf6740 1 client.25951 shutdown 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c717110 msgr2=0x55560c714c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c717110 0x55560c714c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0cb340 tx=0x7fdefc09a3a0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 shutdown_connections 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c727320 0x55560c729740 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdef0c00640 0 client.25951 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c717110 0x55560c714c90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c714750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/2796144047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c70dc10 0x55560c714210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 >> 172.21.15.6:0/2796144047 conn(0x55560c708340 msgr2=0x55560c70a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 shutdown_connections 2024-02-26T06:29:34.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.633+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/2796144047 wait complete. 2024-02-26T06:29:34.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.634+0000 7fdf0cdf6740 20 client.25951 trim_cache size 0 max 16384 2024-02-26T06:29:34.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.649+0000 7fdf0cdf6740 1 Processor -- start 2024-02-26T06:29:34.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.649+0000 7fdf0cdf6740 1 -- start start 2024-02-26T06:29:34.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c716940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c72e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c711470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c3747e0 con 0x55560c717110 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c708f10 con 0x55560c70e530 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c70fe40 con 0x55560c70dc10 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c716940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c716940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36246/0 (socket says 172.21.15.6:36246) 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 -- 172.21.15.6:0/436109473 learned_addr learned my addr 172.21.15.6:0/436109473 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf03400640 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c711470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdefa000640 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c72e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 -- 172.21.15.6:0/436109473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 msgr2=0x55560c72e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c72e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 -- 172.21.15.6:0/436109473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 msgr2=0x55560c711470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c711470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf02a00640 1 -- 172.21.15.6:0/436109473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55560c60fa80 con 0x55560c70dc10 2024-02-26T06:29:34.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.650+0000 7fdf03400640 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c711470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:34.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.651+0000 7fdf02a00640 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c716940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdef40104a0 tx=0x7fdef4024040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9b8349a525f00f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.651+0000 7fdef3400640 1 -- 172.21.15.6:0/436109473 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4004a00 con 0x55560c70dc10 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.651+0000 7fdef3400640 1 -- 172.21.15.6:0/436109473 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdef4004b60 con 0x55560c70dc10 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.651+0000 7fdef3400640 1 -- 172.21.15.6:0/436109473 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4022930 con 0x55560c70dc10 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/436109473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 msgr2=0x55560c716940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c716940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdef40104a0 tx=0x7fdef4024040 comp rx=0 tx=0).stop 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/436109473 shutdown_connections 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c711470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70e530 0x55560c72e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/436109473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70dc10 0x55560c716940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/436109473 >> 172.21.15.6:0/436109473 conn(0x55560c708340 msgr2=0x55560c703380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/436109473 shutdown_connections 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/436109473 wait complete. 2024-02-26T06:29:34.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 -- start start 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.652+0000 7fdf0cdf6740 1 Processor -- start 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 0x55560c71ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70e530 0x55560c71cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c71d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c77d850 con 0x55560c717110 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c77d990 con 0x55560c70dc10 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c77db00 con 0x55560c70e530 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf0cdf6740 10 client.0 _ll_register_callbacks cb 0x7fdef4009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf03400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c71d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf02a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 0x55560c71ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdefa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70e530 0x55560c71cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf02a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 0x55560c71ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56648/0 (socket says 172.21.15.6:56648) 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.653+0000 7fdf02a00640 1 -- 172.21.15.6:0/3450404428 learned_addr learned my addr 172.21.15.6:0/3450404428 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdf02a00640 1 -- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70e530 msgr2=0x55560c71cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdf02a00640 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70e530 0x55560c71cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdf02a00640 1 -- 172.21.15.6:0/3450404428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 msgr2=0x55560c71d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdf02a00640 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c71d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdf02a00640 1 -- 172.21.15.6:0/3450404428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef4030040 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdf02a00640 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 0x55560c71ca10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdef403a0f0 tx=0x7fdef403a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbad909fd1312013 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdef0c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.654+0000 7fdef0c00640 1 -- 172.21.15.6:0/3450404428 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef403a400 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdef4026070 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdef0c00640 1 -- 172.21.15.6:0/3450404428 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fdef403a560 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdef0c00640 1 -- 172.21.15.6:0/3450404428 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef403a720 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 10 client.25979 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55560c635620 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdef0c00640 1 -- 172.21.15.6:0/3450404428 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdef4026070 con 0x55560c70dc10 2024-02-26T06:29:34.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdef0c00640 1 -- 172.21.15.6:0/3450404428 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdef402a030 con 0x55560c70dc10 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 10 client.25979 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 10 client.25979 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 20 client.25979 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c77efb0 0x55560c7813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 4 client.25979 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.655+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 --> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55560c400890 con 0x55560c77efb0 2024-02-26T06:29:34.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdefa000640 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c77efb0 0x55560c7813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdefa000640 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c77efb0 0x55560c7813d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20d7814745735882 server_cookie=7a2991f115b44837 in_seq=0 out_seq=0 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdef0c00640 10 client.25979 ms_handle_connect on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdef0c00640 1 -- 172.21.15.6:0/3450404428 <== mds.4 v2:172.21.15.167:6836/30308127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55560c400890 con 0x55560c77efb0 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdef0c00640 10 client.25979 handle_command_reply: tid=0 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdf0cdf6740 1 client.25979 shutdown 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 msgr2=0x55560c71ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.656+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 0x55560c71ca10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdef403a0f0 tx=0x7fdef403a120 comp rx=0 tx=0).stop 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 shutdown_connections 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:6836/30308127,v1:172.21.15.167:6837/30308127] conn(0x55560c77efb0 0x55560c7813d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55560c717110 0x55560c71d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55560c70e530 0x55560c71cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 --2- 172.21.15.6:0/3450404428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55560c70dc10 0x55560c71ca10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdef0c00640 0 client.25979 ms_handle_reset on v2:172.21.15.167:6836/30308127 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 >> 172.21.15.6:0/3450404428 conn(0x55560c708340 msgr2=0x55560c72e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 shutdown_connections 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 1 -- 172.21.15.6:0/3450404428 wait complete. 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf0cdf6740 20 client.25979 trim_cache size 0 max 16384 2024-02-26T06:29:34.661 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:34.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fdee80a2420 msgr2=0x7fdee80a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf09e00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fdee80a2420 0x7fdee80a48e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf0410aa90 tx=0x7fdeec005b50 comp rx=0 tx=0).stop 2024-02-26T06:29:34.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04109cc0 msgr2=0x7fdf041a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.657+0000 7fdf09e00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04109cc0 0x7fdf041a9e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d5c0 tx=0x7fdefc00da80 comp rx=0 tx=0).stop 2024-02-26T06:29:34.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 shutdown_connections 2024-02-26T06:29:34.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fdee80a2420 0x7fdee80a48e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf04109cc0 0x7fdf041a9e50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf04105360 0x7fdf04109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 --2- 172.21.15.6:0/361937017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf04100290 0x7fdf04109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 >> 172.21.15.6:0/361937017 conn(0x7fdf04077230 msgr2=0x7fdf040fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 shutdown_connections 2024-02-26T06:29:34.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.658+0000 7fdf09e00640 1 -- 172.21.15.6:0/361937017 wait complete. 2024-02-26T06:29:34.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:34 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3518395747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:34.675 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:34.675 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-26T06:29:34.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:34 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3518395747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:34.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:34 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3518395747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.878+0000 7f5412c00640 1 -- 172.21.15.6:0/3452593082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073e70 msgr2=0x7f540c10d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.878+0000 7f5412c00640 1 --2- 172.21.15.6:0/3452593082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073e70 0x7f540c10d280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f802bcd0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 -- 172.21.15.6:0/3452593082 shutdown_connections 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 --2- 172.21.15.6:0/3452593082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 --2- 172.21.15.6:0/3452593082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073e70 0x7f540c10d280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 --2- 172.21.15.6:0/3452593082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f540c073550 0x7f540c073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 -- 172.21.15.6:0/3452593082 >> 172.21.15.6:0/3452593082 conn(0x7f540c0fca30 msgr2=0x7f540c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 -- 172.21.15.6:0/3452593082 shutdown_connections 2024-02-26T06:29:34.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.879+0000 7f5412c00640 1 -- 172.21.15.6:0/3452593082 wait complete. 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 Processor -- start 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 -- start start 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073550 0x7f540c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f540c073e70 0x7f540c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c199e60 con 0x7f540c073e70 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c199fd0 con 0x7f540c10d7c0 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c19a190 con 0x7f540c073550 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5412200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.880+0000 7f5411800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073550 0x7f540c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56662/0 (socket says 172.21.15.6:56662) 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 -- 172.21.15.6:0/741026656 learned_addr learned my addr 172.21.15.6:0/741026656 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5410e00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f540c073e70 0x7f540c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 -- 172.21.15.6:0/741026656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073550 msgr2=0x7f540c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073550 0x7f540c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 -- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f540c073e70 msgr2=0x7f540c19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f540c073e70 0x7f540c19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 -- 172.21.15.6:0/741026656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8009560 con 0x7f540c10d7c0 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412200640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c1996d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f540400eb10 tx=0x7f540400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f540a000640 1 -- 172.21.15.6:0/741026656 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400ce80 con 0x7f540c10d7c0 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f540a000640 1 -- 172.21.15.6:0/741026656 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5404005020 con 0x7f540c10d7c0 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.881+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c19a470 con 0x7f540c10d7c0 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.882+0000 7f540a000640 1 -- 172.21.15.6:0/741026656 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404010700 con 0x7f540c10d7c0 2024-02-26T06:29:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.882+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c1a6540 con 0x7f540c10d7c0 2024-02-26T06:29:34.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.885+0000 7f541721c740 1 Processor -- start 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.885+0000 7f540a000640 1 -- 172.21.15.6:0/741026656 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f54040054e0 con 0x7f540c10d7c0 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.885+0000 7f541721c740 1 -- start start 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f541721c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f541721c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f541721c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f541721c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111a051e0 con 0x55b111a976e0 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f541721c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111a02710 con 0x55b111aa02c0 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f541721c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1119fdef0 con 0x55b111a96dc0 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36274/0 (socket says 172.21.15.6:36274) 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 -- 172.21.15.6:0/3209828637 learned_addr learned my addr 172.21.15.6:0/3209828637 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5411800640 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 -- 172.21.15.6:0/3209828637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 msgr2=0x55b111aa26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 -- 172.21.15.6:0/3209828637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 msgr2=0x55b111a9fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.886+0000 7f5410e00640 1 -- 172.21.15.6:0/3209828637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1119bfdc0 con 0x55b111a96dc0 2024-02-26T06:29:34.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f5410e00640 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a971a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58036aac30947c3e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f540a000640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f53f40a2350 0x7f53f40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f53fea00640 1 -- 172.21.15.6:0/3209828637 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8004b00 con 0x55b111a96dc0 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f53fea00640 1 -- 172.21.15.6:0/3209828637 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f8004510 con 0x55b111a96dc0 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f53fea00640 1 -- 172.21.15.6:0/3209828637 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f80051c0 con 0x55b111a96dc0 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f540a000640 1 -- 172.21.15.6:0/741026656 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f5404014070 con 0x7f540c10d7c0 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.887+0000 7f5411800640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f53f40a2350 0x7f53f40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 -- 172.21.15.6:0/3209828637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 msgr2=0x55b111a971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a971a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f8005e20 comp rx=0 tx=0).stop 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 -- 172.21.15.6:0/3209828637 shutdown_connections 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 --2- 172.21.15.6:0/3209828637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a971a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 -- 172.21.15.6:0/3209828637 >> 172.21.15.6:0/3209828637 conn(0x55b111a914f0 msgr2=0x55b111a918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 -- 172.21.15.6:0/3209828637 shutdown_connections 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f5411800640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f53f40a2350 0x7f53f40a4810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55b111a96890 tx=0x7f5400005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 -- 172.21.15.6:0/3209828637 wait complete. 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.888+0000 7f541721c740 1 -- start start 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 Processor -- start 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111a96dc0 0x55b111a9d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111a9d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111aa02c0 0x55b111a9dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1119a3120 con 0x55b111aa02c0 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1119a2dc0 con 0x55b111a96dc0 2024-02-26T06:29:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11176c1c0 con 0x55b111a976e0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f541721c740 10 client.0 _ll_register_callbacks cb 0x55b1119f2280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f5411800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111a96dc0 0x55b111a9d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f5410e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111a9d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.889+0000 7f5412200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111aa02c0 0x55b111a9dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5411800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111a96dc0 0x55b111a9d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56680/0 (socket says 172.21.15.6:56680) 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5411800640 1 -- 172.21.15.6:0/2399060333 learned_addr learned my addr 172.21.15.6:0/2399060333 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5410e00640 1 -- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111a96dc0 msgr2=0x55b111a9d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5410e00640 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111a96dc0 0x55b111a9d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5410e00640 1 -- 172.21.15.6:0/2399060333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111aa02c0 msgr2=0x55b111a9dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5410e00640 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111aa02c0 0x55b111a9dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5410e00640 1 -- 172.21.15.6:0/2399060333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c1108a0 con 0x55b111a976e0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5412200640 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111aa02c0 0x55b111a9dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f5410e00640 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111a9d850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f802dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e0673fddb4f629 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.890+0000 7f53f3400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:34.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f53f3400640 1 -- 172.21.15.6:0/2399060333 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8005140 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f53f3400640 1 -- 172.21.15.6:0/2399060333 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f53f80052a0 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1116fd7e0 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f53f3400640 1 -- 172.21.15.6:0/2399060333 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53f8004b00 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f541721c740 10 client.25969 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b1119fc570 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f53f3400640 1 -- 172.21.15.6:0/2399060333 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f8059080 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.891+0000 7f53f3400640 1 -- 172.21.15.6:0/2399060333 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f53f803c070 con 0x55b111a976e0 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f541721c740 10 client.25969 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f541721c740 10 client.25969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f541721c740 20 client.25969 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f541721c740 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111ab0020 0x55b111ab2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f541721c740 4 client.25969 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:34.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b11173a720 con 0x55b111ab0020 2024-02-26T06:29:34.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.892+0000 7f5411800640 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111ab0020 0x55b111ab2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f5411800640 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111ab0020 0x55b111ab2410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=386aeee68729bf4c server_cookie=15ed97fbeb743afd in_seq=0 out_seq=0 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f53f3400640 10 client.25969 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f53f3400640 1 -- 172.21.15.6:0/2399060333 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b11173a720 con 0x55b111ab0020 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f53f3400640 10 client.25969 handle_command_reply: tid=0 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f541721c740 1 client.25969 shutdown 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 msgr2=0x55b111a9d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.893+0000 7f541721c740 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111a9d850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f802dac0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 shutdown_connections 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111ab0020 0x55b111ab2410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111aa02c0 0x55b111a9dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111a9d850 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 --2- 172.21.15.6:0/2399060333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111a96dc0 0x55b111a9d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f53f3400640 0 client.25969 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 >> 172.21.15.6:0/2399060333 conn(0x55b111a914f0 msgr2=0x55b111a93240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 shutdown_connections 2024-02-26T06:29:34.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 1 -- 172.21.15.6:0/2399060333 wait complete. 2024-02-26T06:29:34.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.894+0000 7f541721c740 20 client.25969 trim_cache size 0 max 16384 2024-02-26T06:29:34.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 Processor -- start 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 -- start start 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a9d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111a9a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111a9fd70 con 0x55b111a976e0 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111ab75e0 con 0x55b111aa02c0 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f541721c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111aa2c80 con 0x55b111a96dc0 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f5411800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a9d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f5411800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a9d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36282/0 (socket says 172.21.15.6:36282) 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f5410e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.908+0000 7f5411800640 1 -- 172.21.15.6:0/367082534 learned_addr learned my addr 172.21.15.6:0/367082534 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5412200640 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111a9a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5411800640 1 -- 172.21.15.6:0/367082534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 msgr2=0x55b111a9a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5411800640 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111a9a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5411800640 1 -- 172.21.15.6:0/367082534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 msgr2=0x55b111a9dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5411800640 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5411800640 1 -- 172.21.15.6:0/367082534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b111998a80 con 0x55b111a96dc0 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5410e00640 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f5411800640 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a9d860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55b111ab2cb0 tx=0x7f540001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac331891ec91f22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f53fe000640 1 -- 172.21.15.6:0/367082534 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540002e070 con 0x55b111a96dc0 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f53fe000640 1 -- 172.21.15.6:0/367082534 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f540002a890 con 0x55b111a96dc0 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.909+0000 7f53fe000640 1 -- 172.21.15.6:0/367082534 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400025040 con 0x55b111a96dc0 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 -- 172.21.15.6:0/367082534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 msgr2=0x55b111a9d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a9d860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55b111ab2cb0 tx=0x7f540001b000 comp rx=0 tx=0).stop 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 -- 172.21.15.6:0/367082534 shutdown_connections 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111a9a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a976e0 0x55b111a9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 --2- 172.21.15.6:0/367082534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a96dc0 0x55b111a9d860 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 -- 172.21.15.6:0/367082534 >> 172.21.15.6:0/367082534 conn(0x55b111a914f0 msgr2=0x55b111a98640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 -- 172.21.15.6:0/367082534 shutdown_connections 2024-02-26T06:29:34.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.910+0000 7f541721c740 1 -- 172.21.15.6:0/367082534 wait complete. 2024-02-26T06:29:34.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 -- start start 2024-02-26T06:29:34.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 Processor -- start 2024-02-26T06:29:34.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a96dc0 0x55b111aa5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111aa60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111b064c0 con 0x55b111a96dc0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111b06600 con 0x55b111aa02c0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b111b06740 con 0x55b111a976e0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.911+0000 7f541721c740 10 client.0 _ll_register_callbacks cb 0x7f54040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111aa60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5411800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a96dc0 0x55b111aa5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111aa60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36290/0 (socket says 172.21.15.6:36290) 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 -- 172.21.15.6:0/3954821586 learned_addr learned my addr 172.21.15.6:0/3954821586 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5412200640 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 -- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 msgr2=0x55b111aa65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 -- 172.21.15.6:0/3954821586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a96dc0 msgr2=0x55b111aa5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a96dc0 0x55b111aa5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 -- 172.21.15.6:0/3954821586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540001f430 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f5410e00640 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111aa60a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006010 tx=0x7f53f80058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf06aeeae3aaa3b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f53f3400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.912+0000 7f53f3400640 1 -- 172.21.15.6:0/3954821586 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8034cf0 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53f8004510 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f53f3400640 1 -- 172.21.15.6:0/3954821586 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f53f8000e40 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f53f3400640 1 -- 172.21.15.6:0/3954821586 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53f8009780 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 10 client.25975 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b1119be620 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f53f3400640 1 -- 172.21.15.6:0/3954821586 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f8004510 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f53f3400640 1 -- 172.21.15.6:0/3954821586 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f53f803c040 con 0x55b111a976e0 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 10 client.25975 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 10 client.25975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-02-26T06:29:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 20 client.25975 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111b07b40 0x55b111b09f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 4 client.25975 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.913+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 --> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b111789890 con 0x55b111b07b40 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f5411800640 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111b07b40 0x55b111b09f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f5411800640 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111b07b40 0x55b111b09f60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cfef7a35a6c619be server_cookie=5e94a44ee70d5373 in_seq=0 out_seq=0 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f53f3400640 10 client.25975 ms_handle_connect on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f53f3400640 1 -- 172.21.15.6:0/3954821586 <== mds.3 v2:172.21.15.90:6836/3613516978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b111789890 con 0x55b111b07b40 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f53f3400640 10 client.25975 handle_command_reply: tid=0 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f541721c740 1 client.25975 shutdown 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 msgr2=0x55b111aa60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.914+0000 7f541721c740 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111aa60a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006010 tx=0x7f53f80058c0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 shutdown_connections 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:6836/3613516978,v1:172.21.15.90:6837/3613516978] conn(0x55b111b07b40 0x55b111b09f60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b111aa02c0 0x55b111aa65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b111a976e0 0x55b111aa60a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 --2- 172.21.15.6:0/3954821586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b111a96dc0 0x55b111aa5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f53f3400640 0 client.25975 ms_handle_reset on v2:172.21.15.90:6836/3613516978 2024-02-26T06:29:34.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 >> 172.21.15.6:0/3954821586 conn(0x55b111a914f0 msgr2=0x55b111a8fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 shutdown_connections 2024-02-26T06:29:34.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 1 -- 172.21.15.6:0/3954821586 wait complete. 2024-02-26T06:29:34.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f541721c740 20 client.25975 trim_cache size 0 max 16384 2024-02-26T06:29:34.919 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f53f40a2350 msgr2=0x7f53f40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f5412c00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f53f40a2350 0x7f53f40a4810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55b111a96890 tx=0x7f5400005f90 comp rx=0 tx=0).stop 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 msgr2=0x7f540c1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.915+0000 7f5412c00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c1996d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f540400eb10 tx=0x7f540400c2d0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 shutdown_connections 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f53f40a2350 0x7f53f40a4810 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f540c10d7c0 0x7f540c1996d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f540c073e70 0x7f540c19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 --2- 172.21.15.6:0/741026656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c073550 0x7f540c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 >> 172.21.15.6:0/741026656 conn(0x7f540c0fca30 msgr2=0x7f540c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 shutdown_connections 2024-02-26T06:29:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:34.916+0000 7f5412c00640 1 -- 172.21.15.6:0/741026656 wait complete. 2024-02-26T06:29:34.935 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:34.935 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T06:29:35.066 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-02-26T06:29:35.066 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.066 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-26T06:29:35.066 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-26T06:29:35.067 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-02-26T06:29:35.068 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-02-26T06:29:35.082 INFO:tasks.workunit.client.0.smithi006.stderr:+ make -j4 2024-02-26T06:29:35.107 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize -cfi 2024-02-26T06:29:35.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 -- 172.21.15.6:0/1389201994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101d20 msgr2=0x7f4768105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 --2- 172.21.15.6:0/1389201994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101d20 0x7f4768105d60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4754009890 tx=0x7f475402bb70 comp rx=0 tx=0).stop 2024-02-26T06:29:35.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 -- 172.21.15.6:0/1389201994 shutdown_connections 2024-02-26T06:29:35.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 --2- 172.21.15.6:0/1389201994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4768106440 0x7f476810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 --2- 172.21.15.6:0/1389201994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101d20 0x7f4768105d60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 --2- 172.21.15.6:0/1389201994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4768101370 0x7f4768101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 -- 172.21.15.6:0/1389201994 >> 172.21.15.6:0/1389201994 conn(0x7f47680fcbf0 msgr2=0x7f47680ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 -- 172.21.15.6:0/1389201994 shutdown_connections 2024-02-26T06:29:35.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.146+0000 7f476cc00640 1 -- 172.21.15.6:0/1389201994 wait complete. 2024-02-26T06:29:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.147+0000 7f476cc00640 1 Processor -- start 2024-02-26T06:29:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.147+0000 7f476cc00640 1 -- start start 2024-02-26T06:29:35.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f476cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 0x7f476819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f476cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4768106440 0x7f476819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f476cc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47681998f0 0x7f4768199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f476cc00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476819a2e0 con 0x7f47681998f0 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f476cc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476819a450 con 0x7f4768101370 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f476cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476819a610 con 0x7f4768106440 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 0x7f476819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 0x7f476819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56722/0 (socket says 172.21.15.6:56722) 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 -- 172.21.15.6:0/1105411401 learned_addr learned my addr 172.21.15.6:0/1105411401 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765400640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4768106440 0x7f476819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4766800640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47681998f0 0x7f4768199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 -- 172.21.15.6:0/1105411401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4768106440 msgr2=0x7f476819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4768106440 0x7f476819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 -- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47681998f0 msgr2=0x7f4768199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47681998f0 0x7f4768199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.148+0000 7f4765e00640 1 -- 172.21.15.6:0/1105411401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4760009900 con 0x7f4768101370 2024-02-26T06:29:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.149+0000 7f4765e00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 0x7f476819f6a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f475800e9f0 tx=0x7f475800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.149+0000 7f475ea00640 1 -- 172.21.15.6:0/1105411401 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758009720 con 0x7f4768101370 2024-02-26T06:29:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.149+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4754009490 con 0x7f4768101370 2024-02-26T06:29:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.149+0000 7f475ea00640 1 -- 172.21.15.6:0/1105411401 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475800cd80 con 0x7f4768101370 2024-02-26T06:29:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.149+0000 7f475ea00640 1 -- 172.21.15.6:0/1105411401 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758010690 con 0x7f4768101370 2024-02-26T06:29:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.149+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47681a68b0 con 0x7f4768101370 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f475ea00640 1 -- 172.21.15.6:0/1105411401 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f47580046c0 con 0x7f4768101370 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 Processor -- start 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 -- start start 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 0x5644bc1a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a5530 0x5644bc1adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1ae110 0x5644bc1b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc1131e0 con 0x5644bc1ae110 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc110710 con 0x5644bc1a4c10 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f476fba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc10bef0 con 0x5644bc1a5530 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f4765e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a5530 0x5644bc1adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.153+0000 7f4766800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 0x5644bc1a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 0x5644bc1a4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56724/0 (socket says 172.21.15.6:56724) 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4765e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a5530 0x5644bc1adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36310/0 (socket says 172.21.15.6:36310) 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 -- 172.21.15.6:0/2588812784 learned_addr learned my addr 172.21.15.6:0/2588812784 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4765400640 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1ae110 0x5644bc1b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 -- 172.21.15.6:0/2588812784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a5530 msgr2=0x5644bc1adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a5530 0x5644bc1adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 -- 172.21.15.6:0/2588812784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1ae110 msgr2=0x5644bc1b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1ae110 0x5644bc1b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 -- 172.21.15.6:0/2588812784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644bc0cddc0 con 0x5644bc1a4c10 2024-02-26T06:29:35.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f475ea00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f474c0a2350 0x7f474c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.154+0000 7f4766800640 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 0x5644bc1a4ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4760009fd0 tx=0x7f47600095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a765b2216dd24a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f475ea00640 1 -- 172.21.15.6:0/1105411401 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4758014070 con 0x7f4768101370 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f4765e00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f474c0a2350 0x7f474c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f4753000640 1 -- 172.21.15.6:0/2588812784 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760013040 con 0x5644bc1a4c10 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f4753000640 1 -- 172.21.15.6:0/2588812784 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4760003e40 con 0x5644bc1a4c10 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f4753000640 1 -- 172.21.15.6:0/2588812784 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000dc00 con 0x5644bc1a4c10 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f476fba7740 1 -- 172.21.15.6:0/2588812784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 msgr2=0x5644bc1a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f4765e00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f474c0a2350 0x7f474c0a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47580cbd10 tx=0x7f47580d4060 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.155+0000 7f476fba7740 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 0x5644bc1a4ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4760009fd0 tx=0x7f47600095c0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 -- 172.21.15.6:0/2588812784 shutdown_connections 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1ae110 0x5644bc1b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a5530 0x5644bc1adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 --2- 172.21.15.6:0/2588812784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a4c10 0x5644bc1a4ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 -- 172.21.15.6:0/2588812784 >> 172.21.15.6:0/2588812784 conn(0x5644bc19f340 msgr2=0x5644bc19f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 -- 172.21.15.6:0/2588812784 shutdown_connections 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 -- 172.21.15.6:0/2588812784 wait complete. 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 -- start start 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.156+0000 7f476fba7740 1 Processor -- start 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a4c10 0x5644bc1ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 0x5644bc1ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1ae110 0x5644bc1abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc0b1120 con 0x5644bc1a4c10 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc0b0dc0 con 0x5644bc1a5530 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bbe7a1c0 con 0x5644bc1ae110 2024-02-26T06:29:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f476fba7740 10 client.0 _ll_register_callbacks cb 0x5644bc100280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f4765400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 0x5644bc1ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f4766800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a4c10 0x5644bc1ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f4765400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 0x5644bc1ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56734/0 (socket says 172.21.15.6:56734) 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f4765400640 1 -- 172.21.15.6:0/2240796715 learned_addr learned my addr 172.21.15.6:0/2240796715 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.157+0000 7f4766800640 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1ae110 0x5644bc1abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4765400640 1 -- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1ae110 msgr2=0x5644bc1abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4765400640 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1ae110 0x5644bc1abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4765400640 1 -- 172.21.15.6:0/2240796715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a4c10 msgr2=0x5644bc1ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4765400640 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a4c10 0x5644bc1ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4765400640 1 -- 172.21.15.6:0/2240796715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4760009db0 con 0x5644bc1a5530 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4765400640 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 0x5644bc1ab6d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4754000c00 tx=0x7f4754004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2033413809312684 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4747400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4747400640 1 -- 172.21.15.6:0/2240796715 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754034640 con 0x5644bc1a5530 2024-02-26T06:29:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f4747400640 1 -- 172.21.15.6:0/2240796715 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f47540347a0 con 0x5644bc1a5530 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.158+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644bbe0b7e0 con 0x5644bc1a5530 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.159+0000 7f4747400640 1 -- 172.21.15.6:0/2240796715 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4754034960 con 0x5644bc1a5530 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.159+0000 7f476fba7740 10 client.26009 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.159+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5644bc10a570 con 0x5644bc1a5530 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.159+0000 7f4747400640 1 -- 172.21.15.6:0/2240796715 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644bbe0b7e0 con 0x5644bc1a5530 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.159+0000 7f4747400640 1 -- 172.21.15.6:0/2240796715 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4754032b60 con 0x5644bc1a5530 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f476fba7740 10 client.26009 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f476fba7740 10 client.26009 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f476fba7740 20 client.26009 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f476fba7740 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc1be0e0 0x5644bc1c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f476fba7740 4 client.26009 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644bbe48720 con 0x5644bc1be0e0 2024-02-26T06:29:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f4766800640 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc1be0e0 0x5644bc1c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f4766800640 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc1be0e0 0x5644bc1c0500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c162dd677df688f server_cookie=363cab783948f544 in_seq=0 out_seq=0 2024-02-26T06:29:35.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.160+0000 7f4747400640 10 client.26009 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:35.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f4747400640 1 -- 172.21.15.6:0/2240796715 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644bbe48720 con 0x5644bc1be0e0 2024-02-26T06:29:35.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f4747400640 10 client.26009 handle_command_reply: tid=0 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 client.26009 shutdown 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 msgr2=0x5644bc1ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 0x5644bc1ab6d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4754000c00 tx=0x7f4754004270 comp rx=0 tx=0).stop 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 shutdown_connections 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc1be0e0 0x5644bc1c0500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1ae110 0x5644bc1abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.161+0000 7f476fba7740 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1a5530 0x5644bc1ab6d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.162+0000 7f476fba7740 1 --2- 172.21.15.6:0/2240796715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a4c10 0x5644bc1ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.162+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 >> 172.21.15.6:0/2240796715 conn(0x5644bc19f340 msgr2=0x5644bc1a1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.162+0000 7f4747400640 0 client.26009 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.162+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 shutdown_connections 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.162+0000 7f476fba7740 1 -- 172.21.15.6:0/2240796715 wait complete. 2024-02-26T06:29:35.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.162+0000 7f476fba7740 20 client.26009 trim_cache size 0 max 16384 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 Processor -- start 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 -- start start 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc1bace0 con 0x5644bc1a5530 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc1a45e0 con 0x5644bc1ae110 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f476fba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc1c5750 con 0x5644bc1a4c10 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f4765400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f4766800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f4765400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47810/0 (socket says 172.21.15.6:47810) 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f4765400640 1 -- 172.21.15.6:0/287634095 learned_addr learned my addr 172.21.15.6:0/287634095 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.175+0000 7f4766800640 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4765400640 1 -- 172.21.15.6:0/287634095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 msgr2=0x5644bc1ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4765400640 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4765400640 1 -- 172.21.15.6:0/287634095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 msgr2=0x5644bc1a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4765400640 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4765400640 1 -- 172.21.15.6:0/287634095 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644bc0a6a80 con 0x5644bc1a5530 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4766800640 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:35.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4765400640 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1aba30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f475402bbb0 tx=0x7f475402c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f86e368e392d46 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4752600640 1 -- 172.21.15.6:0/287634095 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754003c00 con 0x5644bc1a5530 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4752600640 1 -- 172.21.15.6:0/287634095 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4754003810 con 0x5644bc1a5530 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.176+0000 7f4752600640 1 -- 172.21.15.6:0/287634095 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754032860 con 0x5644bc1a5530 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 -- 172.21.15.6:0/287634095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 msgr2=0x5644bc1aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1aba30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f475402bbb0 tx=0x7f475402c310 comp rx=0 tx=0).stop 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 -- 172.21.15.6:0/287634095 shutdown_connections 2024-02-26T06:29:35.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1aba30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 --2- 172.21.15.6:0/287634095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 -- 172.21.15.6:0/287634095 >> 172.21.15.6:0/287634095 conn(0x5644bc19f340 msgr2=0x5644bc1a5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 -- 172.21.15.6:0/287634095 shutdown_connections 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 -- 172.21.15.6:0/287634095 wait complete. 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.177+0000 7f476fba7740 1 -- start start 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 Processor -- start 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc1b46c0 con 0x5644bc1a5530 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc214630 con 0x5644bc1ae110 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644bc214770 con 0x5644bc1a4c10 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f476fba7740 10 client.0 _ll_register_callbacks cb 0x7f4760009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f4766800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f4766800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36330/0 (socket says 172.21.15.6:36330) 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f4766800640 1 -- 172.21.15.6:0/3206450499 learned_addr learned my addr 172.21.15.6:0/3206450499 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.178+0000 7f4765400640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 -- 172.21.15.6:0/3206450499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 msgr2=0x5644bc1b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 -- 172.21.15.6:0/3206450499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 msgr2=0x5644bc1b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 -- 172.21.15.6:0/3206450499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644bc10a570 con 0x5644bc1a4c10 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4765400640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:35.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4766800640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1b5510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4760006010 tx=0x7f4754009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcec76499d12008 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4747400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4747400640 1 -- 172.21.15.6:0/3206450499 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760013030 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f4747400640 1 -- 172.21.15.6:0/3206450499 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f47600050e0 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.179+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4760003e40 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f4747400640 1 -- 172.21.15.6:0/3206450499 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4760033070 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 10 client.25993 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5644bc0cc620 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f4747400640 1 -- 172.21.15.6:0/3206450499 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4760003e40 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f4747400640 1 -- 172.21.15.6:0/3206450499 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4760039060 con 0x5644bc1a4c10 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 10 client.25993 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 10 client.25993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 20 client.25993 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc215f70 0x5644bc218390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 4 client.25993 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:35.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.180+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 --> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5644bbe97890 con 0x5644bc215f70 2024-02-26T06:29:35.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.181+0000 7f4765400640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc215f70 0x5644bc218390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.181+0000 7f4765400640 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc215f70 0x5644bc218390 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ea3b25e7f9a48ac server_cookie=978d0930fa7cedc in_seq=0 out_seq=0 2024-02-26T06:29:35.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.181+0000 7f4747400640 10 client.25993 ms_handle_connect on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.182+0000 7f4747400640 1 -- 172.21.15.6:0/3206450499 <== mds.2 v2:172.21.15.167:6838/3988004767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+42032 (crc 0 0 0) 0x5644bbe97890 con 0x5644bc215f70 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.182+0000 7f4747400640 10 client.25993 handle_command_reply: tid=0 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.182+0000 7f476fba7740 1 client.25993 shutdown 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.182+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 msgr2=0x5644bc1b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.182+0000 7f476fba7740 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1b5510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4760006010 tx=0x7f4754009780 comp rx=0 tx=0).stop 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 shutdown_connections 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:6838/3988004767,v1:172.21.15.167:6839/3988004767] conn(0x5644bc215f70 0x5644bc218390 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644bc1ae110 0x5644bc1b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5644bc1a5530 0x5644bc1b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 --2- 172.21.15.6:0/3206450499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644bc1a4c10 0x5644bc1b5510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 >> 172.21.15.6:0/3206450499 conn(0x5644bc19f340 msgr2=0x5644bc1a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f4747400640 0 client.25993 ms_handle_reset on v2:172.21.15.167:6838/3988004767 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 shutdown_connections 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 1 -- 172.21.15.6:0/3206450499 wait complete. 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476fba7740 20 client.25993 trim_cache size 0 max 16384 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2024-02-26T06:29:35.187 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 18316392, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628768, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030.out" 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 62197034, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628785, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039" 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 92026321, 2024-02-26T06:29:35.188 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628710, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001.out" 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 108292251, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628469, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/local.config.example" 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 112061791, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628814, 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053.out" 2024-02-26T06:29:35.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 113486076, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628360, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.overlay" 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 137323371, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628817, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055" 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 137872017, 2024-02-26T06:29:35.190 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628721, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007" 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 141329641, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628741, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017" 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 156069734, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628712, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002.out" 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.191 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 163762466, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628802, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047.out" 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 175769419, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628748, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020.out" 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 234878355, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628831, 2024-02-26T06:29:35.192 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062" 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 287326256, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628709, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001" 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 288800201, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628733, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013" 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.193 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 295683950, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628812, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052.out" 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 319913246, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628859, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076" 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 334311658, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628804, 2024-02-26T06:29:35.194 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048.out" 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 338136856, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628813, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053" 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 396340376, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628745, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019" 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.195 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 423853142, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628372, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/check" 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 424427863, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628820, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056.out" 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 483582521, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628796, 2024-02-26T06:29:35.196 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044.out" 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 547693287, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628806, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049.out" 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 586442548, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628774, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033.out" 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 632551621, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628771, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032" 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 633071175, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628724, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008.out" 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 636361469, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628720, 2024-02-26T06:29:35.198 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006.out" 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 674060162, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628707, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/Makefile" 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 678512179, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628744, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018.out" 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.199 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 678824374, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628759, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026" 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 689532280, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627783, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev" 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 691744699, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628718, 2024-02-26T06:29:35.200 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005.out" 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 766784802, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628504, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/soak" 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 827635642, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628775, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034" 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.201 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 849468382, 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628421, 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configs" 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 852348080, 2024-02-26T06:29:35.202 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628714, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003.out" 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 853838275, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628852, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072.out" 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 857377968, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628773, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033" 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.203 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 866594739, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628736, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014.out" 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 872620103, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628755, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024" 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 922567311, 2024-02-26T06:29:35.204 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628424, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configure.ac" 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 925443430, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628795, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044" 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 952228122, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628754, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023.out" 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.205 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 962231016, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628757, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025" 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 977801061, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628794, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043.out" 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 995645947, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628799, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046" 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.206 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 997235297, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628858, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075.out" 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1048130014, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628821, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057" 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1052348872, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628735, 2024-02-26T06:29:35.207 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014" 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1070979188, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628711, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002" 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1080865611, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628363, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/acinclude.m4" 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.208 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1090375221, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628825, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059" 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1110347972, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628769, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031" 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1128289511, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628824, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058.out" 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.209 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1130959844, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628777, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035" 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1149590410, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628354, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makefile" 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1158794738, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628854, 2024-02-26T06:29:35.210 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073.out" 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1188900743, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628362, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/VERSION" 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1195151188, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628357, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.config-sections" 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1229594400, 2024-02-26T06:29:35.211 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628361, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.selftest" 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1231104870, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628837, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065" 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1236666130, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628823, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058" 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.212 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1383916251, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628750, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021.out" 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1384998433, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628850, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071.out" 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1400782946, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628782, 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037.out" 2024-02-26T06:29:35.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1441193810, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628834, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063.out" 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1474512364, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628845, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069" 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1511008161, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628719, 2024-02-26T06:29:35.214 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006" 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1522833119, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628763, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028" 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1638708292, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628853, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073" 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1639519293, 2024-02-26T06:29:35.215 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628770, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031.out" 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1661348996, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628725, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009" 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1688737364, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628828, 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060.out" 2024-02-26T06:29:35.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1702560051, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628842, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067.out" 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1725004008, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628752, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022.out" 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1762571679, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628454, 2024-02-26T06:29:35.217 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lib" 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1792362871, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628860, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076.out" 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1798640160, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627784, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.git" 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.218 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1801113071, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628856, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074.out" 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1818555291, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628761, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027" 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1845898893, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628822, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057.out" 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.219 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1877248374, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628715, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004" 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1886976327, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628740, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016.out" 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1889780498, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628767, 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030" 2024-02-26T06:29:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1895132250, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628500, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/new" 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1935464310, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628846, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069.out" 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1941265834, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628848, 2024-02-26T06:29:35.221 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070.out" 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1961302866, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628847, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070" 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1963978988, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628732, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012.out" 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1978091216, 2024-02-26T06:29:35.222 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628358, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.device-mapper" 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 1992326118, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628503, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/setup" 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2004747725, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628481, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/m4" 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.223 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2019942481, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628758, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025.out" 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2021637483, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628729, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011" 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2030174298, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628832, 2024-02-26T06:29:35.224 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062.out" 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2061785629, 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628350, 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.gitignore" 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2067917218, 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628471, 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ltp" 2024-02-26T06:29:35.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2077488273, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628800, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046.out" 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2079652074, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628351, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/LICENSES" 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2154215930, 2024-02-26T06:29:35.226 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628501, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/randomize.awk" 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2170686636, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628843, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068" 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2211345521, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628722, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007.out" 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.227 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2244070715, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628779, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036" 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2321835299, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628787, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040" 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2347959772, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627782, 2024-02-26T06:29:35.228 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp" 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2348615912, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628801, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047" 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2353754117, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628353, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/MAINTAINERS" 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.229 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2377562565, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628756, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024.out" 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2393254114, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628809, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051" 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2416915674, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628364, 2024-02-26T06:29:35.230 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/build" 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2423245495, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628789, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041" 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2462311212, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628737, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015" 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2472287229, 2024-02-26T06:29:35.231 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628839, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066" 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2505039921, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628749, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021" 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2509848819, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628829, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061" 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.232 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2521831583, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628728, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010.out" 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2524007184, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628835, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064" 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2524624201, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628772, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032.out" 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.233 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2547002971, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628815, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054" 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2565624808, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628706, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests" 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2627089415, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628861, 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077" 2024-02-26T06:29:35.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2637610515, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628731, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012" 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2643519305, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628436, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/include" 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.235 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2671627619, 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628805, 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049" 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2672011196, 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628753, 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023" 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.236 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.237 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.237 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2685319060, 2024-02-26T06:29:35.237 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628808, 2024-02-26T06:29:35.237 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050.out" 2024-02-26T06:29:35.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.237 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.238 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.238 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2699417663, 2024-02-26T06:29:35.238 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628841, 2024-02-26T06:29:35.238 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067" 2024-02-26T06:29:35.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.238 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2704674260, 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628766, 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029.out" 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.239 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2757517703, 2024-02-26T06:29:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628849, 2024-02-26T06:29:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071" 2024-02-26T06:29:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2794139012, 2024-02-26T06:29:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628355, 2024-02-26T06:29:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makepkgs" 2024-02-26T06:29:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.241 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2834978498, 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628783, 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038" 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.242 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2842046898, 2024-02-26T06:29:35.243 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628751, 2024-02-26T06:29:35.243 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022" 2024-02-26T06:29:35.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.243 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.243 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.243 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2921977336, 2024-02-26T06:29:35.244 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628734, 2024-02-26T06:29:35.244 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013.out" 2024-02-26T06:29:35.244 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.244 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.244 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.245 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2925395252, 2024-02-26T06:29:35.245 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628760, 2024-02-26T06:29:35.245 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026.out" 2024-02-26T06:29:35.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.245 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.245 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.246 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2940806395, 2024-02-26T06:29:35.246 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628833, 2024-02-26T06:29:35.246 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063" 2024-02-26T06:29:35.246 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.246 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.246 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3013770514, 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628425, 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/crash" 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3129731237, 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628790, 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041.out" 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.247 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3159454411, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628807, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050" 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3187167317, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628786, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039.out" 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3219029066, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628765, 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029" 2024-02-26T06:29:35.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3245222132, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628862, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077.out" 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3314226695, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628811, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052" 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.249 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3322173887, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628780, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036.out" 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3330329424, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628738, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015.out" 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3340032691, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628857, 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075" 2024-02-26T06:29:35.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3403175914, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628730, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011.out" 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3404331835, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628781, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037" 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3405666053, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628793, 2024-02-26T06:29:35.251 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043" 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3414733101, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628798, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045.out" 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3473834150, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628855, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074" 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.252 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3489433694, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628838, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065.out" 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3490878550, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628792, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042.out" 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3500041607, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628470, 2024-02-26T06:29:35.253 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lsqa.pl" 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3508271772, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628373, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/common" 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3535030606, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628505, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/src" 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.254 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3549218625, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628739, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016" 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3558184329, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628356, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README" 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3566178328, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628776, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034.out" 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.255 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3610607177, 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628836, 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064.out" 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3611525202, 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628717, 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005" 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.256 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3616988945, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628713, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003" 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3617174496, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628840, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066.out" 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3629350984, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628742, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017.out" 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.257 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3645337272, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628819, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056" 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3662343752, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628502, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/release.sh" 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3677847371, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628429, 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/doc" 2024-02-26T06:29:35.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3715162185, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628810, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051.out" 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3715729873, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628803, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048" 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3732305311, 2024-02-26T06:29:35.259 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628851, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072" 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3771860440, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628762, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027.out" 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3851753351, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628743, 2024-02-26T06:29:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018" 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3854277243, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628797, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045" 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3855482270, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628791, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042" 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3858920018, 2024-02-26T06:29:35.261 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628816, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054.out" 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3912674343, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628784, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038.out" 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3934416319, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628716, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004.out" 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.262 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3959568433, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628827, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060" 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 3977738774, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628844, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068.out" 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4012932185, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628778, 2024-02-26T06:29:35.263 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035.out" 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4041466369, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628726, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009.out" 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4044073785, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628723, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008" 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.264 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4054201677, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628727, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010" 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4056164253, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628830, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061.out" 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4068878113, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628746, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019.out" 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.265 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4115528078, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628708, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs" 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4136617481, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628788, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040.out" 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4141831314, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628818, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055.out" 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.266 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4165085247, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628747, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020" 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4175442545, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628826, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059.out" 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4265805029, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628764, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028.out" 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:29:35.267 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:29:35.268 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2024-02-26T06:29:35.268 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4279962065, 2024-02-26T06:29:35.268 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511628359, 2024-02-26T06:29:35.268 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.fuse" 2024-02-26T06:29:35.268 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:29:35.268 INFO:teuthology.orchestra.run.smithi006.stdout:] 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f474c0a2350 msgr2=0x7f474c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476cc00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f474c0a2350 0x7f474c0a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47580cbd10 tx=0x7f47580d4060 comp rx=0 tx=0).stop 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 msgr2=0x7f476819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.183+0000 7f476cc00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 0x7f476819f6a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f475800e9f0 tx=0x7f475800edf0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 shutdown_connections 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f474c0a2350 0x7f474c0a4810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47681998f0 0x7f4768199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4768106440 0x7f476819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 --2- 172.21.15.6:0/1105411401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4768101370 0x7f476819f6a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 >> 172.21.15.6:0/1105411401 conn(0x7f47680fcbf0 msgr2=0x7f47680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 shutdown_connections 2024-02-26T06:29:35.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.184+0000 7f476cc00640 1 -- 172.21.15.6:0/1105411401 wait complete. 2024-02-26T06:29:35.270 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:35.270 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T06:29:35.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 -- 172.21.15.6:0/308800797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4103f40 msgr2=0x7f47a4107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 --2- 172.21.15.6:0/308800797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4103f40 0x7f47a4107f80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4790009930 tx=0x7f479002bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:35.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 -- 172.21.15.6:0/308800797 shutdown_connections 2024-02-26T06:29:35.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 --2- 172.21.15.6:0/308800797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a4108660 0x7f47a4110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 --2- 172.21.15.6:0/308800797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4103f40 0x7f47a4107f80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 --2- 172.21.15.6:0/308800797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4103590 0x7f47a4103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.502+0000 7f47ac200640 1 -- 172.21.15.6:0/308800797 >> 172.21.15.6:0/308800797 conn(0x7f47a40fedd0 msgr2=0x7f47a41011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.503+0000 7f47ac200640 1 -- 172.21.15.6:0/308800797 shutdown_connections 2024-02-26T06:29:35.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.504+0000 7f47ac200640 1 -- 172.21.15.6:0/308800797 wait complete. 2024-02-26T06:29:35.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.505+0000 7f47ac200640 1 Processor -- start 2024-02-26T06:29:35.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.505+0000 7f47ac200640 1 -- start start 2024-02-26T06:29:35.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47ac200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a4103590 0x7f47a419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47ac200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4108660 0x7f47a419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47ac200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 0x7f47a4199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47ac200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a419a2d0 con 0x7f47a4108660 2024-02-26T06:29:35.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47ac200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a419a440 con 0x7f47a4199820 2024-02-26T06:29:35.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47ac200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a419a5b0 con 0x7f47a4103590 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47a8e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4108660 0x7f47a419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47aa200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 0x7f47a4199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47aa200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 0x7f47a4199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56764/0 (socket says 172.21.15.6:56764) 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47aa200640 1 -- 172.21.15.6:0/381470662 learned_addr learned my addr 172.21.15.6:0/381470662 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.506+0000 7f47a9800640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a4103590 0x7f47a419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.507+0000 7f47aa200640 1 -- 172.21.15.6:0/381470662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a4103590 msgr2=0x7f47a419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.507+0000 7f47aa200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a4103590 0x7f47a419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.507+0000 7f47aa200640 1 -- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4108660 msgr2=0x7f47a419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.507+0000 7f47aa200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4108660 0x7f47a419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.507+0000 7f47aa200640 1 -- 172.21.15.6:0/381470662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798009490 con 0x7f47a4199820 2024-02-26T06:29:35.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.508+0000 7f47aa200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 0x7f47a4199cd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f479c00eb10 tx=0x7f479c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.508+0000 7f47a2000640 1 -- 172.21.15.6:0/381470662 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c004b20 con 0x7f47a4199820 2024-02-26T06:29:35.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.508+0000 7f47a2000640 1 -- 172.21.15.6:0/381470662 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f479c004c80 con 0x7f47a4199820 2024-02-26T06:29:35.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.508+0000 7f47a2000640 1 -- 172.21.15.6:0/381470662 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c010650 con 0x7f47a4199820 2024-02-26T06:29:35.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.508+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4790009560 con 0x7f47a4199820 2024-02-26T06:29:35.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.509+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a4075920 con 0x7f47a4199820 2024-02-26T06:29:35.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.514+0000 7f47a2000640 1 -- 172.21.15.6:0/381470662 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f479c005020 con 0x7f47a4199820 2024-02-26T06:29:35.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.514+0000 7f47af1a5740 1 Processor -- start 2024-02-26T06:29:35.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.515+0000 7f47af1a5740 1 -- start start 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb2c10 0x564f17cb2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cbbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cbe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47af1a5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17c211e0 con 0x564f17cbc110 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47af1a5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17c1e710 con 0x564f17cb2c10 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47af1a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17c19ef0 con 0x564f17cb3530 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47a9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cbbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47a9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cbbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36364/0 (socket says 172.21.15.6:36364) 2024-02-26T06:29:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47a9800640 1 -- 172.21.15.6:0/249418327 learned_addr learned my addr 172.21.15.6:0/249418327 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.516+0000 7f47a8e00640 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb2c10 0x564f17cb2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a8e00640 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cbe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a9800640 1 -- 172.21.15.6:0/249418327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb2c10 msgr2=0x564f17cb2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a9800640 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb2c10 0x564f17cb2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a9800640 1 -- 172.21.15.6:0/249418327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 msgr2=0x564f17cbe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a9800640 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cbe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a9800640 1 -- 172.21.15.6:0/249418327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f17bdbdc0 con 0x564f17cb3530 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a2000640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f478c0a2450 0x7f478c0a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.517+0000 7f47a8e00640 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cbe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.518+0000 7f47a2000640 1 -- 172.21.15.6:0/381470662 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f479c014070 con 0x7f47a4199820 2024-02-26T06:29:35.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.518+0000 7f47a8e00640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f478c0a2450 0x7f478c0a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.519+0000 7f47a9800640 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cbbbd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4798009e80 tx=0x7f4798011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4bfec76f57f0c96 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.519+0000 7f4796a00640 1 -- 172.21.15.6:0/249418327 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798019410 con 0x564f17cb3530 2024-02-26T06:29:35.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.520+0000 7f47a8e00640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f478c0a2450 0x7f478c0a4910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4790000c00 tx=0x7f4790042040 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.521+0000 7f4796a00640 1 -- 172.21.15.6:0/249418327 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4798019a50 con 0x564f17cb3530 2024-02-26T06:29:35.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.521+0000 7f4796a00640 1 -- 172.21.15.6:0/249418327 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798019410 con 0x564f17cb3530 2024-02-26T06:29:35.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.522+0000 7f47af1a5740 1 -- 172.21.15.6:0/249418327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 msgr2=0x564f17cbbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.522+0000 7f47af1a5740 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cbbbd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4798009e80 tx=0x7f4798011810 comp rx=0 tx=0).stop 2024-02-26T06:29:35.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.523+0000 7f47af1a5740 1 -- 172.21.15.6:0/249418327 shutdown_connections 2024-02-26T06:29:35.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.523+0000 7f47af1a5740 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cbe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.523+0000 7f47af1a5740 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cbbbd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.523+0000 7f47af1a5740 1 --2- 172.21.15.6:0/249418327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb2c10 0x564f17cb2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.523+0000 7f47af1a5740 1 -- 172.21.15.6:0/249418327 >> 172.21.15.6:0/249418327 conn(0x564f17cad340 msgr2=0x564f17cad720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.523+0000 7f47af1a5740 1 -- 172.21.15.6:0/249418327 shutdown_connections 2024-02-26T06:29:35.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.524+0000 7f47af1a5740 1 -- 172.21.15.6:0/249418327 wait complete. 2024-02-26T06:29:35.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.525+0000 7f47af1a5740 1 -- start start 2024-02-26T06:29:35.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.526+0000 7f47af1a5740 1 Processor -- start 2024-02-26T06:29:35.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.526+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.526+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cb9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.526+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cb9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.526+0000 7f47af1a5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17bbf120 con 0x564f17cbc110 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.526+0000 7f47af1a5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17bbedc0 con 0x564f17cb3530 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47af1a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f179881c0 con 0x564f17cb2c10 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47af1a5740 10 client.0 _ll_register_callbacks cb 0x564f17c0e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47a9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47a9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cb9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36368/0 (socket says 172.21.15.6:36368) 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47a9800640 1 -- 172.21.15.6:0/1719166366 learned_addr learned my addr 172.21.15.6:0/1719166366 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47aa200640 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cb9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47a9800640 1 -- 172.21.15.6:0/1719166366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 msgr2=0x564f17cb9690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.527+0000 7f47a9800640 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cb9690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f47a9800640 1 -- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 msgr2=0x564f17cb9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f47a9800640 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cb9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f47a9800640 1 -- 172.21.15.6:0/1719166366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798009ae0 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f47a9800640 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cb9150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f479800a010 tx=0x7f4798019640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23953690240e1d6c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f478b400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f478b400640 1 -- 172.21.15.6:0/1719166366 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798004020 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f478b400640 1 -- 172.21.15.6:0/1719166366 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f4798004180 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.528+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f179197e0 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f478b400640 1 -- 172.21.15.6:0/1719166366 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4798004340 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f47af1a5740 10 client.26005 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564f17c18570 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f478b400640 1 -- 172.21.15.6:0/1719166366 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564f179197e0 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f478b400640 1 -- 172.21.15.6:0/1719166366 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4798018c60 con 0x564f17cb2c10 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f47af1a5740 10 client.26005 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:35.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.529+0000 7f47af1a5740 10 client.26005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-02-26T06:29:35.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.530+0000 7f47af1a5740 20 client.26005 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:35.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.530+0000 7f47af1a5740 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17ccc120 0x564f17cce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.530+0000 7f47af1a5740 4 client.26005 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:35.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.530+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f17956720 con 0x564f17ccc120 2024-02-26T06:29:35.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.531+0000 7f47a8e00640 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17ccc120 0x564f17cce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.533+0000 7f47a8e00640 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17ccc120 0x564f17cce540 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=467ff7115c6c6cd3 server_cookie=c1a50316ea4abf1f in_seq=0 out_seq=0 2024-02-26T06:29:35.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.533+0000 7f478b400640 10 client.26005 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:35.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.534+0000 7f478b400640 1 -- 172.21.15.6:0/1719166366 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f17956720 con 0x564f17ccc120 2024-02-26T06:29:35.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.534+0000 7f478b400640 10 client.26005 handle_command_reply: tid=0 2024-02-26T06:29:35.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.534+0000 7f47af1a5740 1 client.26005 shutdown 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 msgr2=0x564f17cb9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cb9150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f479800a010 tx=0x7f4798019640 comp rx=0 tx=0).stop 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 shutdown_connections 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17ccc120 0x564f17cce540 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cb9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f478b400640 0 client.26005 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:35.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cb9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 --2- 172.21.15.6:0/1719166366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cb9150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 >> 172.21.15.6:0/1719166366 conn(0x564f17cad340 msgr2=0x564f17caf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 shutdown_connections 2024-02-26T06:29:35.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.535+0000 7f47af1a5740 1 -- 172.21.15.6:0/1719166366 wait complete. 2024-02-26T06:29:35.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.536+0000 7f47af1a5740 20 client.26005 trim_cache size 0 max 16384 2024-02-26T06:29:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.559+0000 7f47af1a5740 1 Processor -- start 2024-02-26T06:29:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.560+0000 7f47af1a5740 1 -- start start 2024-02-26T06:29:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.560+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cb2c10 0x564f17cbb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.560+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cb96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.560+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cbc110 0x564f17cb9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cb96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47af1a5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17cba150 con 0x564f17cb2c10 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47af1a5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17cacea0 con 0x564f17cbc110 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47af1a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17cc8d00 con 0x564f17cb3530 2024-02-26T06:29:35.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cb96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:36374/0 (socket says 172.21.15.6:36374) 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 -- 172.21.15.6:0/4010289342 learned_addr learned my addr 172.21.15.6:0/4010289342 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47aa200640 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cbc110 0x564f17cb9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 -- 172.21.15.6:0/4010289342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cbc110 msgr2=0x564f17cb9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cbc110 0x564f17cb9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 -- 172.21.15.6:0/4010289342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cb2c10 msgr2=0x564f17cbb880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cb2c10 0x564f17cbb880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.561+0000 7f47a8e00640 1 -- 172.21.15.6:0/4010289342 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f17bb4a80 con 0x564f17cb3530 2024-02-26T06:29:35.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.563+0000 7f47a8e00640 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cb96a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4790005ab0 tx=0x7f4790009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a40f96296663d06 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.563+0000 7f4796000640 1 -- 172.21.15.6:0/4010289342 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47900049f0 con 0x564f17cb3530 2024-02-26T06:29:35.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.564+0000 7f4796000640 1 -- 172.21.15.6:0/4010289342 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4790053040 con 0x564f17cb3530 2024-02-26T06:29:35.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.564+0000 7f4796000640 1 -- 172.21.15.6:0/4010289342 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47900049f0 con 0x564f17cb3530 2024-02-26T06:29:35.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.565+0000 7f47af1a5740 1 -- 172.21.15.6:0/4010289342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 msgr2=0x564f17cb96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.565+0000 7f47af1a5740 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cb96a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4790005ab0 tx=0x7f4790009260 comp rx=0 tx=0).stop 2024-02-26T06:29:35.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.566+0000 7f47af1a5740 1 -- 172.21.15.6:0/4010289342 shutdown_connections 2024-02-26T06:29:35.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.566+0000 7f47af1a5740 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cbc110 0x564f17cb9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.566+0000 7f47af1a5740 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb3530 0x564f17cb96a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.566+0000 7f47af1a5740 1 --2- 172.21.15.6:0/4010289342 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cb2c10 0x564f17cbb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.566+0000 7f47af1a5740 1 -- 172.21.15.6:0/4010289342 >> 172.21.15.6:0/4010289342 conn(0x564f17cad340 msgr2=0x564f17cb4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.566+0000 7f47af1a5740 1 -- 172.21.15.6:0/4010289342 shutdown_connections 2024-02-26T06:29:35.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.567+0000 7f47af1a5740 1 -- 172.21.15.6:0/4010289342 wait complete. 2024-02-26T06:29:35.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.567+0000 7f47af1a5740 1 -- start start 2024-02-26T06:29:35.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.568+0000 7f47af1a5740 1 Processor -- start 2024-02-26T06:29:35.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.568+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cc1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.568+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cc1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.568+0000 7f47af1a5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cc2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47af1a5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17d22650 con 0x564f17cbc110 2024-02-26T06:29:35.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47af1a5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17d22790 con 0x564f17cb3530 2024-02-26T06:29:35.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47af1a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f17d228d0 con 0x564f17cb2c10 2024-02-26T06:29:35.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47af1a5740 10 client.0 _ll_register_callbacks cb 0x7f4798009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:35.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47a8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cc1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47a8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cc1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56794/0 (socket says 172.21.15.6:56794) 2024-02-26T06:29:35.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.569+0000 7f47a8e00640 1 -- 172.21.15.6:0/574521982 learned_addr learned my addr 172.21.15.6:0/574521982 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47a9800640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cc1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47a8e00640 1 -- 172.21.15.6:0/574521982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 msgr2=0x564f17cc1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47aa200640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cc2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47a8e00640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cc1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47a8e00640 1 -- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 msgr2=0x564f17cc2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47a8e00640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cc2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.570+0000 7f47a8e00640 1 -- 172.21.15.6:0/574521982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a4110a40 con 0x564f17cb3530 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.571+0000 7f47a9800640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cc1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.571+0000 7f47aa200640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:35.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.571+0000 7f47a8e00640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cc1fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4790036040 tx=0x7f479002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e37e5e456cdc8cb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.571+0000 7f478b400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:35.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.572+0000 7f478b400640 1 -- 172.21.15.6:0/574521982 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790004390 con 0x564f17cb3530 2024-02-26T06:29:35.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.572+0000 7f478b400640 1 -- 172.21.15.6:0/574521982 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f47900409d0 con 0x564f17cb3530 2024-02-26T06:29:35.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.572+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4798019a50 con 0x564f17cb3530 2024-02-26T06:29:35.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.572+0000 7f478b400640 1 -- 172.21.15.6:0/574521982 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4790040b90 con 0x564f17cb3530 2024-02-26T06:29:35.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.572+0000 7f47af1a5740 10 client.26030 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:35.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.572+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564f17bda620 con 0x564f17cb3530 2024-02-26T06:29:35.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.573+0000 7f478b400640 1 -- 172.21.15.6:0/574521982 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4798019a50 con 0x564f17cb3530 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f478b400640 1 -- 172.21.15.6:0/574521982 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f47900039f0 con 0x564f17cb3530 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f47af1a5740 10 client.26030 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f47af1a5740 10 client.26030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f47af1a5740 20 client.26030 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f47af1a5740 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17d23e20 0x564f17d26240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f47af1a5740 4 client.26030 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:35.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.574+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 --> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f179a5890 con 0x564f17d23e20 2024-02-26T06:29:35.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.575+0000 7f47a9800640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17d23e20 0x564f17d26240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.576+0000 7f47a9800640 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17d23e20 0x564f17d26240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=447d73befe7f7f10 server_cookie=5df03e0a9732f723 in_seq=0 out_seq=0 2024-02-26T06:29:35.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.576+0000 7f478b400640 10 client.26030 ms_handle_connect on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:35.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.576+0000 7f478b400640 1 -- 172.21.15.6:0/574521982 <== mds.1 v2:172.21.15.6:6840/3799445989 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f179a5890 con 0x564f17d23e20 2024-02-26T06:29:35.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.576+0000 7f478b400640 10 client.26030 handle_command_reply: tid=0 2024-02-26T06:29:35.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.577+0000 7f47af1a5740 1 client.26030 shutdown 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.577+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 msgr2=0x564f17cc1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.577+0000 7f47af1a5740 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cc1fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4790036040 tx=0x7f479002bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.577+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 shutdown_connections 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.577+0000 7f47af1a5740 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:6840/3799445989,v1:172.21.15.6:6841/3799445989] conn(0x564f17d23e20 0x564f17d26240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f47af1a5740 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564f17cbc110 0x564f17cc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f478b400640 0 client.26030 ms_handle_reset on v2:172.21.15.6:6840/3799445989 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f47af1a5740 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f17cb3530 0x564f17cc1fc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f47af1a5740 1 --2- 172.21.15.6:0/574521982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f17cb2c10 0x564f17cc1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 >> 172.21.15.6:0/574521982 conn(0x564f17cad340 msgr2=0x564f17cabbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 shutdown_connections 2024-02-26T06:29:35.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.578+0000 7f47af1a5740 1 -- 172.21.15.6:0/574521982 wait complete. 2024-02-26T06:29:35.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.579+0000 7f47af1a5740 20 client.26030 trim_cache size 0 max 16384 2024-02-26T06:29:35.579 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f478c0a2450 msgr2=0x7f478c0a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f478c0a2450 0x7f478c0a4910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4790000c00 tx=0x7f4790042040 comp rx=0 tx=0).stop 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 msgr2=0x7f47a4199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 0x7f47a4199cd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f479c00eb10 tx=0x7f479c00c2d0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 shutdown_connections 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f478c0a2450 0x7f478c0a4910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4199820 0x7f47a4199cd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47a4108660 0x7f47a419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 --2- 172.21.15.6:0/381470662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a4103590 0x7f47a419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 >> 172.21.15.6:0/381470662 conn(0x7f47a40fedd0 msgr2=0x7f47a4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.580+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 shutdown_connections 2024-02-26T06:29:35.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.581+0000 7f47ac200640 1 -- 172.21.15.6:0/381470662 wait complete. 2024-02-26T06:29:35.608 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:35 smithi006 ceph-mon[31790]: pgmap v47: 97 pgs: 97 active+clean; 54 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 545 op/s 2024-02-26T06:29:35.612 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T06:29:35.612 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T06:29:35.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:35 smithi167 ceph-mon[36229]: pgmap v47: 97 pgs: 97 active+clean; 54 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 545 op/s 2024-02-26T06:29:35.824 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: putting auxiliary files in '.'. 2024-02-26T06:29:35.824 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file './config.guess' 2024-02-26T06:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 -- 172.21.15.6:0/4070022832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0756b0 msgr2=0x7fc84c075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 --2- 172.21.15.6:0/4070022832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0756b0 0x7fc84c075a90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009930 tx=0x7fc83c02bc30 comp rx=0 tx=0).stop 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 -- 172.21.15.6:0/4070022832 shutdown_connections 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 --2- 172.21.15.6:0/4070022832 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c111fc0 0x7fc84c1143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 --2- 172.21.15.6:0/4070022832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 0x7fc84c111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 --2- 172.21.15.6:0/4070022832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0756b0 0x7fc84c075a90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 -- 172.21.15.6:0/4070022832 >> 172.21.15.6:0/4070022832 conn(0x7fc84c0fedd0 msgr2=0x7fc84c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 -- 172.21.15.6:0/4070022832 shutdown_connections 2024-02-26T06:29:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.837+0000 7fc851200640 1 -- 172.21.15.6:0/4070022832 wait complete. 2024-02-26T06:29:35.838 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file './config.sub' 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.838+0000 7fc851200640 1 Processor -- start 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.838+0000 7fc851200640 1 -- start start 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc851200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 0x7fc84c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc851200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 0x7fc84c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc851200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c111fc0 0x7fc84c1a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc851200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1a2cc0 con 0x7fc84c0756b0 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc851200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1a2e30 con 0x7fc84c075fd0 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc851200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1a2ff0 con 0x7fc84c111fc0 2024-02-26T06:29:35.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc849e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 0x7fc84c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc849e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 0x7fc84c19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47862/0 (socket says 172.21.15.6:47862) 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc849e00640 1 -- 172.21.15.6:0/579088336 learned_addr learned my addr 172.21.15.6:0/579088336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc849400640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 0x7fc84c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc84a800640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c111fc0 0x7fc84c1a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.839+0000 7fc849e00640 1 -- 172.21.15.6:0/579088336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c111fc0 msgr2=0x7fc84c1a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc849e00640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c111fc0 0x7fc84c1a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc849e00640 1 -- 172.21.15.6:0/579088336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 msgr2=0x7fc84c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc849e00640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 0x7fc84c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc849e00640 1 -- 172.21.15.6:0/579088336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c009560 con 0x7fc84c0756b0 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc849400640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 0x7fc84c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:35.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc849e00640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 0x7fc84c19de50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009900 tx=0x7fc83c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc842a00640 1 -- 172.21.15.6:0/579088336 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0042d0 con 0x7fc84c0756b0 2024-02-26T06:29:35.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c1a3270 con 0x7fc84c0756b0 2024-02-26T06:29:35.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.840+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c1aab50 con 0x7fc84c0756b0 2024-02-26T06:29:35.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.841+0000 7fc842a00640 1 -- 172.21.15.6:0/579088336 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc83c004430 con 0x7fc84c0756b0 2024-02-26T06:29:35.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.841+0000 7fc842a00640 1 -- 172.21.15.6:0/579088336 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c003af0 con 0x7fc84c0756b0 2024-02-26T06:29:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 Processor -- start 2024-02-26T06:29:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 -- start start 2024-02-26T06:29:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc842a00640 1 -- 172.21.15.6:0/579088336 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 27) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fc83c003c50 con 0x7fc84c0756b0 2024-02-26T06:29:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655246fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655246611e0 con 0x5655246f3530 2024-02-26T06:29:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56552465e710 con 0x5655246f2c10 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.844+0000 7fc8541b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x565524659ef0 con 0x5655246fc110 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc849e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655246fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56824/0 (socket says 172.21.15.6:56824) 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc849e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655246fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47870/0 (socket says 172.21.15.6:47870) 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 -- 172.21.15.6:0/3275090287 learned_addr learned my addr 172.21.15.6:0/3275090287 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc849400640 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 -- 172.21.15.6:0/3275090287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 msgr2=0x5655246fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 -- 172.21.15.6:0/3275090287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 msgr2=0x5655246fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655246fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc84a800640 1 -- 172.21.15.6:0/3275090287 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56552461bdc0 con 0x5655246f2c10 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.845+0000 7fc849400640 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.846+0000 7fc84a800640 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246f2ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc844009f30 tx=0x7fc8440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=418b56793f8833ce server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.846+0000 7fc837400640 1 -- 172.21.15.6:0/3275090287 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844004d20 con 0x5655246f2c10 2024-02-26T06:29:35.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.846+0000 7fc837400640 1 -- 172.21.15.6:0/3275090287 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc844004e80 con 0x5655246f2c10 2024-02-26T06:29:35.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.846+0000 7fc842a00640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc8300a23d0 0x7fc8300a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.846+0000 7fc837400640 1 -- 172.21.15.6:0/3275090287 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844005280 con 0x5655246f2c10 2024-02-26T06:29:35.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.846+0000 7fc842a00640 1 -- 172.21.15.6:0/579088336 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fc83c0f0140 con 0x7fc84c0756b0 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc849e00640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc8300a23d0 0x7fc8300a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 -- 172.21.15.6:0/3275090287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 msgr2=0x5655246f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246f2ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc844009f30 tx=0x7fc8440118d0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 -- 172.21.15.6:0/3275090287 shutdown_connections 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655246fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3275090287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246f2ff0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 -- 172.21.15.6:0/3275090287 >> 172.21.15.6:0/3275090287 conn(0x5655246ed340 msgr2=0x5655246ed720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 -- 172.21.15.6:0/3275090287 shutdown_connections 2024-02-26T06:29:35.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc8541b0740 1 -- 172.21.15.6:0/3275090287 wait complete. 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.847+0000 7fc849e00640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc8300a23d0 0x7fc8300a4890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0ed800 tx=0x7fc83c0f3b90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 -- start start 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 Processor -- start 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f2c10 0x5655246f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 0x5655246f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655245ff120 con 0x5655246f2c10 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655245fedc0 con 0x5655246f3530 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.848+0000 7fc8541b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655243c81c0 con 0x5655246fc110 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc8541b0740 10 client.0 _ll_register_callbacks cb 0x56552464e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 0x5655246f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 0x5655246f9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56838/0 (socket says 172.21.15.6:56838) 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 -- 172.21.15.6:0/3197409605 learned_addr learned my addr 172.21.15.6:0/3197409605 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc84a800640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f2c10 0x5655246f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc84a800640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 -- 172.21.15.6:0/3197409605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 msgr2=0x5655246f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 -- 172.21.15.6:0/3197409605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f2c10 msgr2=0x5655246f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f2c10 0x5655246f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc849400640 1 -- 172.21.15.6:0/3197409605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844009be0 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.849+0000 7fc84a800640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc849400640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 0x5655246f9650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc838012d20 tx=0x7fc83800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab9b0a6ff165897 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc82be00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc82be00640 1 -- 172.21.15.6:0/3197409605 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838017070 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc82be00640 1 -- 172.21.15.6:0/3197409605 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fc838012110 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565524396720 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc82be00640 1 -- 172.21.15.6:0/3197409605 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8380122d0 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc8541b0740 10 client.26042 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x565524658570 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.850+0000 7fc82be00640 1 -- 172.21.15.6:0/3197409605 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565524396720 con 0x5655246f3530 2024-02-26T06:29:35.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc82be00640 1 -- 172.21.15.6:0/3197409605 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc838010430 con 0x5655246f3530 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc8541b0740 10 client.26042 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc8541b0740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc8541b0740 20 client.26042 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56552470c2b0 0x56552470e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc8541b0740 4 client.26042 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5655243e5890 con 0x56552470c2b0 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.851+0000 7fc84a800640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56552470c2b0 0x56552470e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.852+0000 7fc84a800640 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56552470c2b0 0x56552470e6d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bc4b91c319f690 server_cookie=2cf846400bbb571 in_seq=0 out_seq=0 2024-02-26T06:29:35.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.852+0000 7fc82be00640 10 client.26042 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:35.855 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file './install-sh' 2024-02-26T06:29:35.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.852+0000 7fc82be00640 1 -- 172.21.15.6:0/3197409605 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5655243e5890 con 0x56552470c2b0 2024-02-26T06:29:35.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.852+0000 7fc82be00640 10 client.26042 handle_command_reply: tid=0 2024-02-26T06:29:35.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.852+0000 7fc8541b0740 1 client.26042 shutdown 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 msgr2=0x5655246f9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 0x5655246f9650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc838012d20 tx=0x7fc83800f790 comp rx=0 tx=0).stop 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 shutdown_connections 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x56552470c2b0 0x56552470e6d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246fc110 0x5655246f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f3530 0x5655246f9650 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc82be00640 0 client.26042 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 --2- 172.21.15.6:0/3197409605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f2c10 0x5655246f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 >> 172.21.15.6:0/3197409605 conn(0x5655246ed340 msgr2=0x5655246ef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 shutdown_connections 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 1 -- 172.21.15.6:0/3197409605 wait complete. 2024-02-26T06:29:35.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.853+0000 7fc8541b0740 20 client.26042 trim_cache size 0 max 16384 2024-02-26T06:29:35.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:35 smithi090 ceph-mon[35681]: pgmap v47: 97 pgs: 97 active+clean; 54 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 545 op/s 2024-02-26T06:29:35.865 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file './ltmain.sh' 2024-02-26T06:29:35.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.866+0000 7fc8541b0740 1 Processor -- start 2024-02-26T06:29:35.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.866+0000 7fc8541b0740 1 -- start start 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246fb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f3530 0x5655246f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246fc110 0x5655246f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc8541b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655246fede0 con 0x5655246fc110 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc8541b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655246f1d00 con 0x5655246f2c10 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc8541b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655246f25e0 con 0x5655246f3530 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246fb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56848/0 (socket says 172.21.15.6:56848) 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 -- 172.21.15.6:0/471299007 learned_addr learned my addr 172.21.15.6:0/471299007 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246fc110 0x5655246f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc849400640 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f3530 0x5655246f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 -- 172.21.15.6:0/471299007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f3530 msgr2=0x5655246f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f3530 0x5655246f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 -- 172.21.15.6:0/471299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246fc110 msgr2=0x5655246f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246fc110 0x5655246f68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.867+0000 7fc84a800640 1 -- 172.21.15.6:0/471299007 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5655245f4a80 con 0x5655246f2c10 2024-02-26T06:29:35.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.868+0000 7fc84a800640 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246fb840 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc844006010 tx=0x7fc8380128a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ccb24cdc68e6cbc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.868+0000 7fc836a00640 1 -- 172.21.15.6:0/471299007 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440042e0 con 0x5655246f2c10 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.868+0000 7fc836a00640 1 -- 172.21.15.6:0/471299007 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc84402e070 con 0x5655246f2c10 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.868+0000 7fc836a00640 1 -- 172.21.15.6:0/471299007 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440286a0 con 0x5655246f2c10 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 -- 172.21.15.6:0/471299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 msgr2=0x5655246fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246fb840 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc844006010 tx=0x7fc8380128a0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 -- 172.21.15.6:0/471299007 shutdown_connections 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246fc110 0x5655246f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f3530 0x5655246f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 --2- 172.21.15.6:0/471299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246f2c10 0x5655246fb840 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 -- 172.21.15.6:0/471299007 >> 172.21.15.6:0/471299007 conn(0x5655246ed340 msgr2=0x5655246f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 -- 172.21.15.6:0/471299007 shutdown_connections 2024-02-26T06:29:35.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 -- 172.21.15.6:0/471299007 wait complete. 2024-02-26T06:29:35.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.869+0000 7fc8541b0740 1 -- start start 2024-02-26T06:29:35.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 Processor -- start 2024-02-26T06:29:35.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f2c10 0x565524701d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655247022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246fc110 0x5655247627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x565524702a70 con 0x5655246f3530 2024-02-26T06:29:35.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x565524702be0 con 0x5655246fc110 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x565524762d00 con 0x5655246f2c10 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.870+0000 7fc8541b0740 10 client.0 _ll_register_callbacks cb 0x7fc844009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc84a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246fc110 0x5655247627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655247022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655247022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47894/0 (socket says 172.21.15.6:47894) 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 -- 172.21.15.6:0/598780500 learned_addr learned my addr 172.21.15.6:0/598780500 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc84a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246fc110 0x5655247627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:56862/0 (socket says 172.21.15.6:56862) 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc84a800640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f2c10 0x565524701d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 -- 172.21.15.6:0/598780500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f2c10 msgr2=0x565524701d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f2c10 0x565524701d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 -- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246fc110 msgr2=0x5655247627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246fc110 0x5655247627c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 -- 172.21.15.6:0/598780500 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844030040 con 0x5655246f3530 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc84a800640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f2c10 0x565524701d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:35.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc849400640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655247022b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5655246fa6e0 tx=0x7fc83800fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a1aadbdd2bb56c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.871+0000 7fc82be00640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.872+0000 7fc82be00640 1 -- 172.21.15.6:0/598780500 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838017030 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.872+0000 7fc82be00640 1 -- 172.21.15.6:0/598780500 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7fc8380109d0 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.872+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5655243597e0 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.872+0000 7fc82be00640 1 -- 172.21.15.6:0/598780500 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc838010b90 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.872+0000 7fc8541b0740 10 client.15900 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.872+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56552461a620 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc82be00640 1 -- 172.21.15.6:0/598780500 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc83803a020 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc82be00640 1 -- 172.21.15.6:0/598780500 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc838003a00 con 0x5655246f3530 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc8541b0740 10 client.15900 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc8541b0740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc8541b0740 20 client.15900 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc8541b0740 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565524764600 0x565524766a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:35.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc8541b0740 4 client.15900 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T06:29:35.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5655243484e0 con 0x565524764600 2024-02-26T06:29:35.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.873+0000 7fc84a800640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565524764600 0x565524766a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc84a800640 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565524764600 0x565524766a20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3531951c3fa3699a server_cookie=a4d1eaf18fea3490 in_seq=0 out_seq=0 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc82be00640 10 client.15900 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc82be00640 1 -- 172.21.15.6:0/598780500 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5655243484e0 con 0x565524764600 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc82be00640 10 client.15900 handle_command_reply: tid=0 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 client.15900 shutdown 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 msgr2=0x5655247022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655247022b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5655246fa6e0 tx=0x7fc83800fd00 comp rx=0 tx=0).stop 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 shutdown_connections 2024-02-26T06:29:35.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x565524764600 0x565524766a20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5655246fc110 0x5655247627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5655246f3530 0x5655247022b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 --2- 172.21.15.6:0/598780500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5655246f2c10 0x565524701d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 >> 172.21.15.6:0/598780500 conn(0x5655246ed340 msgr2=0x5655246ebbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.874+0000 7fc82be00640 0 client.15900 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 shutdown_connections 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc8541b0740 1 -- 172.21.15.6:0/598780500 wait complete. 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc8541b0740 20 client.15900 trim_cache size 0 max 16384 2024-02-26T06:29:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc8300a23d0 msgr2=0x7fc8300a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc851200640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc8300a23d0 0x7fc8300a4890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0ed800 tx=0x7fc83c0f3b90 comp rx=0 tx=0).stop 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 msgr2=0x7fc84c19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.875+0000 7fc851200640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 0x7fc84c19de50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009900 tx=0x7fc83c02c5d0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 shutdown_connections 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fc8300a23d0 0x7fc8300a4890 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c111fc0 0x7fc84c1a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc84c075fd0 0x7fc84c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 --2- 172.21.15.6:0/579088336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84c0756b0 0x7fc84c19de50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:35.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 >> 172.21.15.6:0/579088336 conn(0x7fc84c0fedd0 msgr2=0x7fc84c1136e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 shutdown_connections 2024-02-26T06:29:35.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:29:35.876+0000 7fc851200640 1 -- 172.21.15.6:0/579088336 wait complete. 2024-02-26T06:29:35.895 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 18316392, 'ino': 1099511628768, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030.out'}, {'damage_type': 'backtrace', 'id': 62197034, 'ino': 1099511628785, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039'}, {'damage_type': 'backtrace', 'id': 92026321, 'ino': 1099511628710, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001.out'}, {'damage_type': 'backtrace', 'id': 108292251, 'ino': 1099511628469, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/local.config.example'}, {'damage_type': 'backtrace', 'id': 112061791, 'ino': 1099511628814, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053.out'}, {'damage_type': 'backtrace', 'id': 113486076, 'ino': 1099511628360, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.overlay'}, {'damage_type': 'backtrace', 'id': 137323371, 'ino': 1099511628817, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055'}, {'damage_type': 'backtrace', 'id': 137872017, 'ino': 1099511628721, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007'}, {'damage_type': 'backtrace', 'id': 141329641, 'ino': 1099511628741, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017'}, {'damage_type': 'backtrace', 'id': 156069734, 'ino': 1099511628712, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002.out'}, {'damage_type': 'backtrace', 'id': 163762466, 'ino': 1099511628802, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047.out'}, {'damage_type': 'backtrace', 'id': 175769419, 'ino': 1099511628748, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020.out'}, {'damage_type': 'backtrace', 'id': 234878355, 'ino': 1099511628831, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062'}, {'damage_type': 'backtrace', 'id': 287326256, 'ino': 1099511628709, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/001'}, {'damage_type': 'backtrace', 'id': 288800201, 'ino': 1099511628733, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013'}, {'damage_type': 'backtrace', 'id': 295683950, 'ino': 1099511628812, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052.out'}, {'damage_type': 'backtrace', 'id': 319913246, 'ino': 1099511628859, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076'}, {'damage_type': 'backtrace', 'id': 334311658, 'ino': 1099511628804, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048.out'}, {'damage_type': 'backtrace', 'id': 338136856, 'ino': 1099511628813, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/053'}, {'damage_type': 'backtrace', 'id': 396340376, 'ino': 1099511628745, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019'}, {'damage_type': 'backtrace', 'id': 423853142, 'ino': 1099511628372, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/check'}, {'damage_type': 'backtrace', 'id': 424427863, 'ino': 1099511628820, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056.out'}, {'damage_type': 'backtrace', 'id': 483582521, 'ino': 1099511628796, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044.out'}, {'damage_type': 'backtrace', 'id': 547693287, 'ino': 1099511628806, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049.out'}, {'damage_type': 'backtrace', 'id': 586442548, 'ino': 1099511628774, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033.out'}, {'damage_type': 'backtrace', 'id': 632551621, 'ino': 1099511628771, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032'}, {'damage_type': 'backtrace', 'id': 633071175, 'ino': 1099511628724, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008.out'}, {'damage_type': 'backtrace', 'id': 636361469, 'ino': 1099511628720, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006.out'}, {'damage_type': 'backtrace', 'id': 674060162, 'ino': 1099511628707, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/Makefile'}, {'damage_type': 'backtrace', 'id': 678512179, 'ino': 1099511628744, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018.out'}, {'damage_type': 'backtrace', 'id': 678824374, 'ino': 1099511628759, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026'}, {'damage_type': 'backtrace', 'id': 689532280, 'ino': 1099511627783, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev'}, {'damage_type': 'backtrace', 'id': 691744699, 'ino': 1099511628718, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005.out'}, {'damage_type': 'backtrace', 'id': 766784802, 'ino': 1099511628504, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/soak'}, {'damage_type': 'backtrace', 'id': 827635642, 'ino': 1099511628775, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034'}, {'damage_type': 'backtrace', 'id': 849468382, 'ino': 1099511628421, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configs'}, {'damage_type': 'backtrace', 'id': 852348080, 'ino': 1099511628714, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003.out'}, {'damage_type': 'backtrace', 'id': 853838275, 'ino': 1099511628852, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072.out'}, {'damage_type': 'backtrace', 'id': 857377968, 'ino': 1099511628773, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/033'}, {'damage_type': 'backtrace', 'id': 866594739, 'ino': 1099511628736, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014.out'}, {'damage_type': 'backtrace', 'id': 872620103, 'ino': 1099511628755, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024'}, {'damage_type': 'backtrace', 'id': 922567311, 'ino': 1099511628424, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/configure.ac'}, {'damage_type': 'backtrace', 'id': 925443430, 'ino': 1099511628795, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/044'}, {'damage_type': 'backtrace', 'id': 952228122, 'ino': 1099511628754, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023.out'}, {'damage_type': 'backtrace', 'id': 962231016, 'ino': 1099511628757, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025'}, {'damage_type': 'backtrace', 'id': 977801061, 'ino': 1099511628794, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043.out'}, {'damage_type': 'backtrace', 'id': 995645947, 'ino': 1099511628799, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046'}, {'damage_type': 'backtrace', 'id': 997235297, 'ino': 1099511628858, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075.out'}, {'damage_type': 'backtrace', 'id': 1048130014, 'ino': 1099511628821, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057'}, {'damage_type': 'backtrace', 'id': 1052348872, 'ino': 1099511628735, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/014'}, {'damage_type': 'backtrace', 'id': 1070979188, 'ino': 1099511628711, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/002'}, {'damage_type': 'backtrace', 'id': 1080865611, 'ino': 1099511628363, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/acinclude.m4'}, {'damage_type': 'backtrace', 'id': 1090375221, 'ino': 1099511628825, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059'}, {'damage_type': 'backtrace', 'id': 1110347972, 'ino': 1099511628769, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031'}, {'damage_type': 'backtrace', 'id': 1128289511, 'ino': 1099511628824, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058.out'}, {'damage_type': 'backtrace', 'id': 1130959844, 'ino': 1099511628777, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035'}, {'damage_type': 'backtrace', 'id': 1149590410, 'ino': 1099511628354, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makefile'}, {'damage_type': 'backtrace', 'id': 1158794738, 'ino': 1099511628854, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073.out'}, {'damage_type': 'backtrace', 'id': 1188900743, 'ino': 1099511628362, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/VERSION'}, {'damage_type': 'backtrace', 'id': 1195151188, 'ino': 1099511628357, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.config-sections'}, {'damage_type': 'backtrace', 'id': 1229594400, 'ino': 1099511628361, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.selftest'}, {'damage_type': 'backtrace', 'id': 1231104870, 'ino': 1099511628837, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065'}, {'damage_type': 'backtrace', 'id': 1236666130, 'ino': 1099511628823, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/058'}, {'damage_type': 'backtrace', 'id': 1383916251, 'ino': 1099511628750, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021.out'}, {'damage_type': 'backtrace', 'id': 1384998433, 'ino': 1099511628850, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071.out'}, {'damage_type': 'backtrace', 'id': 1400782946, 'ino': 1099511628782, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037.out'}, {'damage_type': 'backtrace', 'id': 1441193810, 'ino': 1099511628834, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063.out'}, {'damage_type': 'backtrace', 'id': 1474512364, 'ino': 1099511628845, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069'}, {'damage_type': 'backtrace', 'id': 1511008161, 'ino': 1099511628719, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/006'}, {'damage_type': 'backtrace', 'id': 1522833119, 'ino': 1099511628763, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028'}, {'damage_type': 'backtrace', 'id': 1638708292, 'ino': 1099511628853, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/073'}, {'damage_type': 'backtrace', 'id': 1639519293, 'ino': 1099511628770, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/031.out'}, {'damage_type': 'backtrace', 'id': 1661348996, 'ino': 1099511628725, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009'}, {'damage_type': 'backtrace', 'id': 1688737364, 'ino': 1099511628828, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060.out'}, {'damage_type': 'backtrace', 'id': 1702560051, 'ino': 1099511628842, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067.out'}, {'damage_type': 'backtrace', 'id': 1725004008, 'ino': 1099511628752, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022.out'}, {'damage_type': 'backtrace', 'id': 1762571679, 'ino': 1099511628454, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lib'}, {'damage_type': 'backtrace', 'id': 1792362871, 'ino': 1099511628860, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/076.out'}, {'damage_type': 'backtrace', 'id': 1798640160, 'ino': 1099511627784, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.git'}, {'damage_type': 'backtrace', 'id': 1801113071, 'ino': 1099511628856, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074.out'}, {'damage_type': 'backtrace', 'id': 1818555291, 'ino': 1099511628761, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027'}, {'damage_type': 'backtrace', 'id': 1845898893, 'ino': 1099511628822, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/057.out'}, {'damage_type': 'backtrace', 'id': 1877248374, 'ino': 1099511628715, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004'}, {'damage_type': 'backtrace', 'id': 1886976327, 'ino': 1099511628740, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016.out'}, {'damage_type': 'backtrace', 'id': 1889780498, 'ino': 1099511628767, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/030'}, {'damage_type': 'backtrace', 'id': 1895132250, 'ino': 1099511628500, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/new'}, {'damage_type': 'backtrace', 'id': 1935464310, 'ino': 1099511628846, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/069.out'}, {'damage_type': 'backtrace', 'id': 1941265834, 'ino': 1099511628848, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070.out'}, {'damage_type': 'backtrace', 'id': 1961302866, 'ino': 1099511628847, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/070'}, {'damage_type': 'backtrace', 'id': 1963978988, 'ino': 1099511628732, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012.out'}, {'damage_type': 'backtrace', 'id': 1978091216, 'ino': 1099511628358, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.device-mapper'}, {'damage_type': 'backtrace', 'id': 1992326118, 'ino': 1099511628503, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/setup'}, {'damage_type': 'backtrace', 'id': 2004747725, 'ino': 1099511628481, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/m4'}, {'damage_type': 'backtrace', 'id': 2019942481, 'ino': 1099511628758, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/025.out'}, {'damage_type': 'backtrace', 'id': 2021637483, 'ino': 1099511628729, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011'}, {'damage_type': 'backtrace', 'id': 2030174298, 'ino': 1099511628832, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/062.out'}, {'damage_type': 'backtrace', 'id': 2061785629, 'ino': 1099511628350, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/.gitignore'}, {'damage_type': 'backtrace', 'id': 2067917218, 'ino': 1099511628471, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/ltp'}, {'damage_type': 'backtrace', 'id': 2077488273, 'ino': 1099511628800, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/046.out'}, {'damage_type': 'backtrace', 'id': 2079652074, 'ino': 1099511628351, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/LICENSES'}, {'damage_type': 'backtrace', 'id': 2154215930, 'ino': 1099511628501, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/randomize.awk'}, {'damage_type': 'backtrace', 'id': 2170686636, 'ino': 1099511628843, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068'}, {'damage_type': 'backtrace', 'id': 2211345521, 'ino': 1099511628722, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/007.out'}, {'damage_type': 'backtrace', 'id': 2244070715, 'ino': 1099511628779, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036'}, {'damage_type': 'backtrace', 'id': 2321835299, 'ino': 1099511628787, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040'}, {'damage_type': 'backtrace', 'id': 2347959772, 'ino': 1099511627782, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp'}, {'damage_type': 'backtrace', 'id': 2348615912, 'ino': 1099511628801, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/047'}, {'damage_type': 'backtrace', 'id': 2353754117, 'ino': 1099511628353, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/MAINTAINERS'}, {'damage_type': 'backtrace', 'id': 2377562565, 'ino': 1099511628756, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/024.out'}, {'damage_type': 'backtrace', 'id': 2393254114, 'ino': 1099511628809, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051'}, {'damage_type': 'backtrace', 'id': 2416915674, 'ino': 1099511628364, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/build'}, {'damage_type': 'backtrace', 'id': 2423245495, 'ino': 1099511628789, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041'}, {'damage_type': 'backtrace', 'id': 2462311212, 'ino': 1099511628737, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015'}, {'damage_type': 'backtrace', 'id': 2472287229, 'ino': 1099511628839, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066'}, {'damage_type': 'backtrace', 'id': 2505039921, 'ino': 1099511628749, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/021'}, {'damage_type': 'backtrace', 'id': 2509848819, 'ino': 1099511628829, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061'}, {'damage_type': 'backtrace', 'id': 2521831583, 'ino': 1099511628728, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010.out'}, {'damage_type': 'backtrace', 'id': 2524007184, 'ino': 1099511628835, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064'}, {'damage_type': 'backtrace', 'id': 2524624201, 'ino': 1099511628772, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/032.out'}, {'damage_type': 'backtrace', 'id': 2547002971, 'ino': 1099511628815, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054'}, {'damage_type': 'backtrace', 'id': 2565624808, 'ino': 1099511628706, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests'}, {'damage_type': 'backtrace', 'id': 2627089415, 'ino': 1099511628861, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077'}, {'damage_type': 'backtrace', 'id': 2637610515, 'ino': 1099511628731, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/012'}, {'damage_type': 'backtrace', 'id': 2643519305, 'ino': 1099511628436, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/include'}, {'damage_type': 'backtrace', 'id': 2671627619, 'ino': 1099511628805, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/049'}, {'damage_type': 'backtrace', 'id': 2672011196, 'ino': 1099511628753, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/023'}, {'damage_type': 'backtrace', 'id': 2685319060, 'ino': 1099511628808, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050.out'}, {'damage_type': 'backtrace', 'id': 2699417663, 'ino': 1099511628841, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/067'}, {'damage_type': 'backtrace', 'id': 2704674260, 'ino': 1099511628766, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029.out'}, {'damage_type': 'backtrace', 'id': 2757517703, 'ino': 1099511628849, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/071'}, {'damage_type': 'backtrace', 'id': 2794139012, 'ino': 1099511628355, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/Makepkgs'}, {'damage_type': 'backtrace', 'id': 2834978498, 'ino': 1099511628783, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038'}, {'damage_type': 'backtrace', 'id': 2842046898, 'ino': 1099511628751, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/022'}, {'damage_type': 'backtrace', 'id': 2921977336, 'ino': 1099511628734, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/013.out'}, {'damage_type': 'backtrace', 'id': 2925395252, 'ino': 1099511628760, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/026.out'}, {'damage_type': 'backtrace', 'id': 2940806395, 'ino': 1099511628833, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/063'}, {'damage_type': 'backtrace', 'id': 3013770514, 'ino': 1099511628425, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/crash'}, {'damage_type': 'backtrace', 'id': 3129731237, 'ino': 1099511628790, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/041.out'}, {'damage_type': 'backtrace', 'id': 3159454411, 'ino': 1099511628807, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/050'}, {'damage_type': 'backtrace', 'id': 3187167317, 'ino': 1099511628786, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/039.out'}, {'damage_type': 'backtrace', 'id': 3219029066, 'ino': 1099511628765, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/029'}, {'damage_type': 'backtrace', 'id': 3245222132, 'ino': 1099511628862, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/077.out'}, {'damage_type': 'backtrace', 'id': 3314226695, 'ino': 1099511628811, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/052'}, {'damage_type': 'backtrace', 'id': 3322173887, 'ino': 1099511628780, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/036.out'}, {'damage_type': 'backtrace', 'id': 3330329424, 'ino': 1099511628738, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/015.out'}, {'damage_type': 'backtrace', 'id': 3340032691, 'ino': 1099511628857, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/075'}, {'damage_type': 'backtrace', 'id': 3403175914, 'ino': 1099511628730, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/011.out'}, {'damage_type': 'backtrace', 'id': 3404331835, 'ino': 1099511628781, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/037'}, {'damage_type': 'backtrace', 'id': 3405666053, 'ino': 1099511628793, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/043'}, {'damage_type': 'backtrace', 'id': 3414733101, 'ino': 1099511628798, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045.out'}, {'damage_type': 'backtrace', 'id': 3473834150, 'ino': 1099511628855, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/074'}, {'damage_type': 'backtrace', 'id': 3489433694, 'ino': 1099511628838, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/065.out'}, {'damage_type': 'backtrace', 'id': 3490878550, 'ino': 1099511628792, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042.out'}, {'damage_type': 'backtrace', 'id': 3500041607, 'ino': 1099511628470, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/lsqa.pl'}, {'damage_type': 'backtrace', 'id': 3508271772, 'ino': 1099511628373, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/common'}, {'damage_type': 'backtrace', 'id': 3535030606, 'ino': 1099511628505, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/src'}, {'damage_type': 'backtrace', 'id': 3549218625, 'ino': 1099511628739, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/016'}, {'damage_type': 'backtrace', 'id': 3558184329, 'ino': 1099511628356, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README'}, {'damage_type': 'backtrace', 'id': 3566178328, 'ino': 1099511628776, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/034.out'}, {'damage_type': 'backtrace', 'id': 3610607177, 'ino': 1099511628836, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/064.out'}, {'damage_type': 'backtrace', 'id': 3611525202, 'ino': 1099511628717, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/005'}, {'damage_type': 'backtrace', 'id': 3616988945, 'ino': 1099511628713, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/003'}, {'damage_type': 'backtrace', 'id': 3617174496, 'ino': 1099511628840, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/066.out'}, {'damage_type': 'backtrace', 'id': 3629350984, 'ino': 1099511628742, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/017.out'}, {'damage_type': 'backtrace', 'id': 3645337272, 'ino': 1099511628819, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/056'}, {'damage_type': 'backtrace', 'id': 3662343752, 'ino': 1099511628502, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/release.sh'}, {'damage_type': 'backtrace', 'id': 3677847371, 'ino': 1099511628429, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/doc'}, {'damage_type': 'backtrace', 'id': 3715162185, 'ino': 1099511628810, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/051.out'}, {'damage_type': 'backtrace', 'id': 3715729873, 'ino': 1099511628803, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/048'}, {'damage_type': 'backtrace', 'id': 3732305311, 'ino': 1099511628851, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/072'}, {'damage_type': 'backtrace', 'id': 3771860440, 'ino': 1099511628762, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/027.out'}, {'damage_type': 'backtrace', 'id': 3851753351, 'ino': 1099511628743, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/018'}, {'damage_type': 'backtrace', 'id': 3854277243, 'ino': 1099511628797, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/045'}, {'damage_type': 'backtrace', 'id': 3855482270, 'ino': 1099511628791, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/042'}, {'damage_type': 'backtrace', 'id': 3858920018, 'ino': 1099511628816, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/054.out'}, {'damage_type': 'backtrace', 'id': 3912674343, 'ino': 1099511628784, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/038.out'}, {'damage_type': 'backtrace', 'id': 3934416319, 'ino': 1099511628716, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/004.out'}, {'damage_type': 'backtrace', 'id': 3959568433, 'ino': 1099511628827, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/060'}, {'damage_type': 'backtrace', 'id': 3977738774, 'ino': 1099511628844, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/068.out'}, {'damage_type': 'backtrace', 'id': 4012932185, 'ino': 1099511628778, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/035.out'}, {'damage_type': 'backtrace', 'id': 4041466369, 'ino': 1099511628726, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/009.out'}, {'damage_type': 'backtrace', 'id': 4044073785, 'ino': 1099511628723, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/008'}, {'damage_type': 'backtrace', 'id': 4054201677, 'ino': 1099511628727, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/010'}, {'damage_type': 'backtrace', 'id': 4056164253, 'ino': 1099511628830, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/061.out'}, {'damage_type': 'backtrace', 'id': 4068878113, 'ino': 1099511628746, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/019.out'}, {'damage_type': 'backtrace', 'id': 4115528078, 'ino': 1099511628708, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs'}, {'damage_type': 'backtrace', 'id': 4136617481, 'ino': 1099511628788, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/040.out'}, {'damage_type': 'backtrace', 'id': 4141831314, 'ino': 1099511628818, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/055.out'}, {'damage_type': 'backtrace', 'id': 4165085247, 'ino': 1099511628747, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/020'}, {'damage_type': 'backtrace', 'id': 4175442545, 'ino': 1099511628826, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/059.out'}, {'damage_type': 'backtrace', 'id': 4265805029, 'ino': 1099511628764, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/tests/btrfs/028.out'}, {'damage_type': 'backtrace', 'id': 4279962065, 'ino': 1099511628359, 'path': '/volumes/qa/sv_1/838d5287-f0ed-4bca-aa45-4ad0fce85018/client.0/tmp/xfstests-dev/README.fuse'}] 2024-02-26T06:29:35.896 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-02-26T06:29:35.950 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-02-26T06:29:35.950 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-02-26T06:29:35.968 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-02-26T06:29:35.987 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-02-26T06:29:36.007 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-02-26T06:29:36.028 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-02-26T06:29:36.056 INFO:tasks.workunit.client.0.smithi006.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-02-26T06:29:36.061 INFO:tasks.workunit.client.0.smithi006.stdout:cp include/install-sh . 2024-02-26T06:29:36.104 INFO:tasks.workunit.client.0.smithi006.stdout:aclocal -I m4 2024-02-26T06:29:37.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:37 smithi006 ceph-mon[31790]: pgmap v48: 97 pgs: 97 active+clean; 54 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 535 op/s 2024-02-26T06:29:37.734 INFO:tasks.workunit.client.0.smithi006.stdout:autoheader 2024-02-26T06:29:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:37 smithi167 ceph-mon[36229]: pgmap v48: 97 pgs: 97 active+clean; 54 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 535 op/s 2024-02-26T06:29:37.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:37 smithi090 ceph-mon[35681]: pgmap v48: 97 pgs: 97 active+clean; 54 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 535 op/s 2024-02-26T06:29:38.210 INFO:tasks.workunit.client.0.smithi006.stdout:autoconf 2024-02-26T06:29:38.515 INFO:tasks.workunit.client.0.smithi006.stdout:./configure \ 2024-02-26T06:29:38.515 INFO:tasks.workunit.client.0.smithi006.stdout: --libexecdir=/usr/lib \ 2024-02-26T06:29:38.516 INFO:tasks.workunit.client.0.smithi006.stdout: --exec_prefix=/var/lib 2024-02-26T06:29:38.934 INFO:tasks.workunit.client.0.smithi006.stdout:checking build system type... x86_64-pc-linux-gnu 2024-02-26T06:29:38.934 INFO:tasks.workunit.client.0.smithi006.stdout:checking host system type... x86_64-pc-linux-gnu 2024-02-26T06:29:38.939 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to print strings... printf 2024-02-26T06:29:38.939 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc... gcc 2024-02-26T06:29:39.135 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether the C compiler works... yes 2024-02-26T06:29:39.135 INFO:tasks.workunit.client.0.smithi006.stdout:checking for C compiler default output file name... a.out 2024-02-26T06:29:39.227 INFO:tasks.workunit.client.0.smithi006.stdout:checking for suffix of executables... 2024-02-26T06:29:39.323 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are cross compiling... no 2024-02-26T06:29:39.402 INFO:tasks.workunit.client.0.smithi006.stdout:checking for suffix of object files... o 2024-02-26T06:29:39.464 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are using the GNU C compiler... yes 2024-02-26T06:29:39.531 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc accepts -g... yes 2024-02-26T06:29:39.664 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to accept ISO C89... none needed 2024-02-26T06:29:39.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:39 smithi006 ceph-mon[31790]: pgmap v49: 97 pgs: 97 active+clean; 58 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 622 op/s 2024-02-26T06:29:39.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:39 smithi006 ceph-mon[31790]: mgrmap e28: x(active, since 92s), standbys: z, y 2024-02-26T06:29:39.678 INFO:tasks.workunit.client.0.smithi006.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-02-26T06:29:39.690 INFO:tasks.workunit.client.0.smithi006.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-02-26T06:29:39.695 INFO:tasks.workunit.client.0.smithi006.stdout:checking for egrep... /usr/bin/grep -E 2024-02-26T06:29:39.700 INFO:tasks.workunit.client.0.smithi006.stdout:checking for fgrep... /usr/bin/grep -F 2024-02-26T06:29:39.715 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ld used by gcc... /usr/bin/ld 2024-02-26T06:29:39.723 INFO:tasks.workunit.client.0.smithi006.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-02-26T06:29:39.741 INFO:tasks.workunit.client.0.smithi006.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-02-26T06:29:39.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:39 smithi167 ceph-mon[36229]: pgmap v49: 97 pgs: 97 active+clean; 58 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 622 op/s 2024-02-26T06:29:39.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:39 smithi167 ceph-mon[36229]: mgrmap e28: x(active, since 92s), standbys: z, y 2024-02-26T06:29:39.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:39 smithi090 ceph-mon[35681]: pgmap v49: 97 pgs: 97 active+clean; 58 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 622 op/s 2024-02-26T06:29:39.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:39 smithi090 ceph-mon[35681]: mgrmap e28: x(active, since 92s), standbys: z, y 2024-02-26T06:29:40.803 INFO:tasks.workunit.client.0.smithi006.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-02-26T06:29:40.803 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether ln -s works... yes 2024-02-26T06:29:40.825 INFO:tasks.workunit.client.0.smithi006.stdout:checking the maximum length of command line arguments... 1572864 2024-02-26T06:29:40.836 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-02-26T06:29:40.837 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-02-26T06:29:40.837 INFO:tasks.workunit.client.0.smithi006.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-02-26T06:29:40.838 INFO:tasks.workunit.client.0.smithi006.stdout:checking for objdump... objdump 2024-02-26T06:29:40.839 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to recognize dependent libraries... pass_all 2024-02-26T06:29:40.841 INFO:tasks.workunit.client.0.smithi006.stdout:checking for dlltool... no 2024-02-26T06:29:40.841 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-02-26T06:29:40.843 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ar... ar 2024-02-26T06:29:41.000 INFO:tasks.workunit.client.0.smithi006.stdout:checking for archiver @FILE support... @ 2024-02-26T06:29:41.001 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strip... strip 2024-02-26T06:29:41.003 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ranlib... ranlib 2024-02-26T06:29:41.004 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gawk... gawk 2024-02-26T06:29:41.248 INFO:tasks.workunit.client.0.smithi006.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-02-26T06:29:41.254 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sysroot... no 2024-02-26T06:29:41.277 INFO:tasks.workunit.client.0.smithi006.stdout:checking for a working dd... /usr/bin/dd 2024-02-26T06:29:41.298 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-02-26T06:29:41.364 INFO:tasks.workunit.client.0.smithi006.stdout:checking for mt... no 2024-02-26T06:29:41.387 INFO:tasks.workunit.client.0.smithi006.stdout:checking if : is a manifest tool... no 2024-02-26T06:29:41.537 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to run the C preprocessor... gcc -E 2024-02-26T06:29:41.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:41 smithi006 ceph-mon[31790]: pgmap v50: 97 pgs: 97 active+clean; 58 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 470 op/s 2024-02-26T06:29:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 -- 172.21.15.6:0/1876324658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073520 msgr2=0x7faa54073900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 --2- 172.21.15.6:0/1876324658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073520 0x7faa54073900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa44009900 tx=0x7faa4402bcc0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 -- 172.21.15.6:0/1876324658 shutdown_connections 2024-02-26T06:29:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 --2- 172.21.15.6:0/1876324658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa54112170 0x7faa54114560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 --2- 172.21.15.6:0/1876324658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073e40 0x7faa54111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 --2- 172.21.15.6:0/1876324658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073520 0x7faa54073900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 -- 172.21.15.6:0/1876324658 >> 172.21.15.6:0/1876324658 conn(0x7faa540fcc30 msgr2=0x7faa540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 -- 172.21.15.6:0/1876324658 shutdown_connections 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.719+0000 7faa5a600640 1 -- 172.21.15.6:0/1876324658 wait complete. 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.720+0000 7faa5a600640 1 Processor -- start 2024-02-26T06:29:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 -- start start 2024-02-26T06:29:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073520 0x7faa5419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 0x7faa5419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa54112170 0x7faa541a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541a2d90 con 0x7faa54073e40 2024-02-26T06:29:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541a2f00 con 0x7faa54112170 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa5a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541a30c0 con 0x7faa54073520 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa4be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 0x7faa5419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa4be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 0x7faa5419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53912/0 (socket says 172.21.15.6:53912) 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa4be00640 1 -- 172.21.15.6:0/3147048689 learned_addr learned my addr 172.21.15.6:0/3147048689 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa53400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073520 0x7faa5419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.721+0000 7faa53e00640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa54112170 0x7faa541a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa53400640 1 -- 172.21.15.6:0/3147048689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa54112170 msgr2=0x7faa541a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa53400640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa54112170 0x7faa541a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa53400640 1 -- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 msgr2=0x7faa5419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa53400640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 0x7faa5419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa53400640 1 -- 172.21.15.6:0/3147048689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44009560 con 0x7faa54073520 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa4be00640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 0x7faa5419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa53400640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073520 0x7faa5419ded0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa44036040 tx=0x7faa4402c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.722+0000 7faa50c00640 1 -- 172.21.15.6:0/3147048689 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44004350 con 0x7faa54073520 2024-02-26T06:29:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.723+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa541a3340 con 0x7faa54073520 2024-02-26T06:29:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.723+0000 7faa50c00640 1 -- 172.21.15.6:0/3147048689 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa440044b0 con 0x7faa54073520 2024-02-26T06:29:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.723+0000 7faa50c00640 1 -- 172.21.15.6:0/3147048689 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44003d30 con 0x7faa54073520 2024-02-26T06:29:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.723+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa541aac20 con 0x7faa54073520 2024-02-26T06:29:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.726+0000 7faa5d638740 1 Processor -- start 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.726+0000 7faa5d638740 1 -- start start 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.726+0000 7faa50c00640 1 -- 172.21.15.6:0/3147048689 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7faa44004df0 con 0x7faa54073520 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa5d638740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 0x555a0c002840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa5d638740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c003200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa5d638740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c00e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa5d638740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bfbe5f0 con 0x555a0c002d80 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa5d638740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bf70890 con 0x555a0c00c300 2024-02-26T06:29:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa5d638740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bf6ddc0 con 0x555a0c003af0 2024-02-26T06:29:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 0x555a0c002840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c003200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 0x555a0c002840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:54772/0 (socket says 172.21.15.6:54772) 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 -- 172.21.15.6:0/232460127 learned_addr learned my addr 172.21.15.6:0/232460127 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa4be00640 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c00e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 -- 172.21.15.6:0/232460127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 msgr2=0x555a0c00e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c00e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 -- 172.21.15.6:0/232460127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 msgr2=0x555a0c003200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c003200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.727+0000 7faa53e00640 1 -- 172.21.15.6:0/232460127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0beeaad0 con 0x555a0c003af0 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa53e00640 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 0x555a0c002840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009f30 tx=0x7faa4c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d8ec4a99a021da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa48c00640 1 -- 172.21.15.6:0/232460127 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c004d20 con 0x555a0c003af0 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa48c00640 1 -- 172.21.15.6:0/232460127 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa4c004e80 con 0x555a0c003af0 2024-02-26T06:29:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa48c00640 1 -- 172.21.15.6:0/232460127 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c005220 con 0x555a0c003af0 2024-02-26T06:29:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa50c00640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7faa380a2220 0x7faa380a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa50c00640 1 -- 172.21.15.6:0/3147048689 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7faa440f0610 con 0x7faa54073520 2024-02-26T06:29:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.728+0000 7faa53400640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7faa380a2220 0x7faa380a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 -- 172.21.15.6:0/232460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 msgr2=0x555a0c002840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 0x555a0c002840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009f30 tx=0x7faa4c0118d0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa53400640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7faa380a2220 0x7faa380a46e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x555a0c004250 tx=0x7faa440f60e0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 -- 172.21.15.6:0/232460127 shutdown_connections 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c00e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c003200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 --2- 172.21.15.6:0/232460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c003af0 0x555a0c002840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 -- 172.21.15.6:0/232460127 >> 172.21.15.6:0/232460127 conn(0x555a0bfca480 msgr2=0x555a0bffda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 -- 172.21.15.6:0/232460127 shutdown_connections 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.729+0000 7faa5d638740 1 -- 172.21.15.6:0/232460127 wait complete. 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 -- start start 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 Processor -- start 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c0092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 0x555a0c009800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c009d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bf695a0 con 0x555a0c002d80 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bf0eb20 con 0x555a0c003af0 2024-02-26T06:29:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.730+0000 7faa5d638740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bf0e7c0 con 0x555a0c00c300 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa5d638740 10 client.0 _ll_register_callbacks cb 0x555a0bf22520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 0x555a0c009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa53e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c0092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 0x555a0c009800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38708/0 (socket says 172.21.15.6:38708) 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 -- 172.21.15.6:0/1772218428 learned_addr learned my addr 172.21.15.6:0/1772218428 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa53e00640 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c009d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 -- 172.21.15.6:0/1772218428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 msgr2=0x555a0c009d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c009d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 -- 172.21.15.6:0/1772218428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 msgr2=0x555a0c0092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c0092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.731+0000 7faa4be00640 1 -- 172.21.15.6:0/1772218428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c009be0 con 0x555a0c003af0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa4be00640 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 0x555a0c009800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faa3c012d50 tx=0x7faa3c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c13c7dc9d1ba88 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa41600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa41600640 1 -- 172.21.15.6:0/1772218428 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c017070 con 0x555a0c003af0 2024-02-26T06:29:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa41600640 1 -- 172.21.15.6:0/1772218428 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7faa3c0120d0 con 0x555a0c003af0 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a0bf3bca0 con 0x555a0c003af0 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa41600640 1 -- 172.21.15.6:0/1772218428 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa3c012290 con 0x555a0c003af0 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa5d638740 10 client.26060 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.732+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555a0bf67c20 con 0x555a0c003af0 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa41600640 1 -- 172.21.15.6:0/1772218428 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555a0bf3bca0 con 0x555a0c003af0 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa41600640 1 -- 172.21.15.6:0/1772218428 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7faa3c010690 con 0x555a0c003af0 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa5d638740 10 client.26060 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa5d638740 10 client.26060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa5d638740 20 client.26060 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa5d638740 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c01c3b0 0x555a0c01e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa5d638740 4 client.26060 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.733+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a0bf5fcb0 con 0x555a0c01c3b0 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.734+0000 7faa53e00640 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c01c3b0 0x555a0c01e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.734+0000 7faa53e00640 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c01c3b0 0x555a0c01e7d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194fbfa4a01f3050 server_cookie=41d58effcfe95e7b in_seq=0 out_seq=0 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.734+0000 7faa41600640 10 client.26060 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa41600640 1 -- 172.21.15.6:0/1772218428 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a0bf5fcb0 con 0x555a0c01c3b0 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa41600640 10 client.26060 handle_command_reply: tid=0 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 client.26060 shutdown 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 msgr2=0x555a0c009800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 0x555a0c009800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faa3c012d50 tx=0x7faa3c00f790 comp rx=0 tx=0).stop 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 shutdown_connections 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c01c3b0 0x555a0c01e7d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c009d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c003af0 0x555a0c009800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 --2- 172.21.15.6:0/1772218428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c002d80 0x555a0c0092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa41600640 0 client.26060 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 >> 172.21.15.6:0/1772218428 conn(0x555a0bfca480 msgr2=0x555a0bfff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.735+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 shutdown_connections 2024-02-26T06:29:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.736+0000 7faa5d638740 1 -- 172.21.15.6:0/1772218428 wait complete. 2024-02-26T06:29:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.736+0000 7faa5d638740 20 client.26060 trim_cache size 0 max 16384 2024-02-26T06:29:41.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:41 smithi167 ceph-mon[36229]: pgmap v50: 97 pgs: 97 active+clean; 58 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 470 op/s 2024-02-26T06:29:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 Processor -- start 2024-02-26T06:29:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 -- start start 2024-02-26T06:29:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c002d80 0x555a0c009b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0bffcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c013160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bcb1e60 con 0x555a0c003af0 2024-02-26T06:29:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0c00ef40 con 0x555a0c00c300 2024-02-26T06:29:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.751+0000 7faa5d638740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0bff9340 con 0x555a0c002d80 2024-02-26T06:29:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0bffcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0bffcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53940/0 (socket says 172.21.15.6:53940) 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 -- 172.21.15.6:0/4246126551 learned_addr learned my addr 172.21.15.6:0/4246126551 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa53e00640 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c002d80 0x555a0c009b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 -- 172.21.15.6:0/4246126551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c002d80 msgr2=0x555a0c009b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c002d80 0x555a0c009b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa53e00640 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c013160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 -- 172.21.15.6:0/4246126551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 msgr2=0x555a0c013160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c013160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 -- 172.21.15.6:0/4246126551 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0bf04500 con 0x555a0c003af0 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa53e00640 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c013160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa4be00640 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0bffcb90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa3c000c00 tx=0x7faa3c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f06e0d33d488dd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.752+0000 7faa43e00640 1 -- 172.21.15.6:0/4246126551 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c017040 con 0x555a0c003af0 2024-02-26T06:29:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa43e00640 1 -- 172.21.15.6:0/4246126551 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa3c005040 con 0x555a0c003af0 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa43e00640 1 -- 172.21.15.6:0/4246126551 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0122d0 con 0x555a0c003af0 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 -- 172.21.15.6:0/4246126551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 msgr2=0x555a0bffcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0bffcb90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa3c000c00 tx=0x7faa3c009d70 comp rx=0 tx=0).stop 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 -- 172.21.15.6:0/4246126551 shutdown_connections 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c00c300 0x555a0c013160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0bffcb90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 --2- 172.21.15.6:0/4246126551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c002d80 0x555a0c009b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.753+0000 7faa5d638740 1 -- 172.21.15.6:0/4246126551 >> 172.21.15.6:0/4246126551 conn(0x555a0bfca480 msgr2=0x555a0bffc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.754+0000 7faa5d638740 1 -- 172.21.15.6:0/4246126551 shutdown_connections 2024-02-26T06:29:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.754+0000 7faa5d638740 1 -- 172.21.15.6:0/4246126551 wait complete. 2024-02-26T06:29:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.754+0000 7faa5d638740 1 -- start start 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.754+0000 7faa5d638740 1 Processor -- start 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c002d80 0x555a0c0122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0c012830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0c072de0 con 0x555a0c003af0 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0c072f20 con 0x555a0c002d80 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0c073060 con 0x555a0c00c300 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa5d638740 10 client.0 _ll_register_callbacks cb 0x7faa3c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa53e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c002d80 0x555a0c0122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa53e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c002d80 0x555a0c0122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38730/0 (socket says 172.21.15.6:38730) 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa53e00640 1 -- 172.21.15.6:0/3397268292 learned_addr learned my addr 172.21.15.6:0/3397268292 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa53e00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.755+0000 7faa4be00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0c012830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa4be00640 1 -- 172.21.15.6:0/3397268292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 msgr2=0x555a0c0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa4be00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c0728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa4be00640 1 -- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c002d80 msgr2=0x555a0c0122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa4be00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c002d80 0x555a0c0122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa4be00640 1 -- 172.21.15.6:0/3397268292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0bf67c20 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa4be00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0c012830 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x555a0bff24f0 tx=0x7faa3c010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f4a26d87d65aeb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa41600640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.756+0000 7faa41600640 1 -- 172.21.15.6:0/3397268292 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c017070 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa41600640 1 -- 172.21.15.6:0/3397268292 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7faa3c0053c0 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a0bf8ed00 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa41600640 1 -- 172.21.15.6:0/3397268292 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa3c0055c0 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 10 client.15918 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555a0bf2a290 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa41600640 1 -- 172.21.15.6:0/3397268292 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555a0bf8ed00 con 0x555a0c003af0 2024-02-26T06:29:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa41600640 1 -- 172.21.15.6:0/3397268292 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7faa3c02d640 con 0x555a0c003af0 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 10 client.15918 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 20 client.15918 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c074390 0x555a0c0767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 4 client.15918 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.757+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a0bc69600 con 0x555a0c074390 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.758+0000 7faa53e00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c074390 0x555a0c0767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.758+0000 7faa53e00640 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c074390 0x555a0c0767b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3e63cdcb1e0378 server_cookie=5ec5dbfaa4c5f95 in_seq=0 out_seq=0 2024-02-26T06:29:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.758+0000 7faa41600640 10 client.15918 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa41600640 1 -- 172.21.15.6:0/3397268292 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a0bc69600 con 0x555a0c074390 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa41600640 10 client.15918 handle_command_reply: tid=0 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 client.15918 shutdown 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 msgr2=0x555a0c012830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0c012830 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x555a0bff24f0 tx=0x7faa3c010980 comp rx=0 tx=0).stop 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 shutdown_connections 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x555a0c074390 0x555a0c0767b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a0c00c300 0x555a0c0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a0c003af0 0x555a0c012830 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 --2- 172.21.15.6:0/3397268292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a0c002d80 0x555a0c0122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 >> 172.21.15.6:0/3397268292 conn(0x555a0bfca480 msgr2=0x555a0bffcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.767+0000 7faa41600640 0 client.15918 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.768+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 shutdown_connections 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.768+0000 7faa5d638740 1 -- 172.21.15.6:0/3397268292 wait complete. 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.768+0000 7faa5d638740 20 client.15918 trim_cache size 0 max 16384 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:29:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7faa380a2220 msgr2=0x7faa380a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7faa380a2220 0x7faa380a46e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x555a0c004250 tx=0x7faa440f60e0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073520 msgr2=0x7faa5419ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073520 0x7faa5419ded0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa44036040 tx=0x7faa4402c710 comp rx=0 tx=0).stop 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 shutdown_connections 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7faa380a2220 0x7faa380a46e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa54112170 0x7faa541a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa54073e40 0x7faa5419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 --2- 172.21.15.6:0/3147048689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa54073520 0x7faa5419ded0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 >> 172.21.15.6:0/3147048689 conn(0x7faa540fcc30 msgr2=0x7faa540fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 shutdown_connections 2024-02-26T06:29:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:41.769+0000 7faa5a600640 1 -- 172.21.15.6:0/3147048689 wait complete. 2024-02-26T06:29:41.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:41 smithi090 ceph-mon[35681]: pgmap v50: 97 pgs: 97 active+clean; 58 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 470 op/s 2024-02-26T06:29:41.931 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ANSI C header files... yes 2024-02-26T06:29:42.015 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/types.h... yes 2024-02-26T06:29:42.111 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/stat.h... yes 2024-02-26T06:29:42.207 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdlib.h... yes 2024-02-26T06:29:42.305 INFO:tasks.workunit.client.0.smithi006.stdout:checking for string.h... yes 2024-02-26T06:29:42.407 INFO:tasks.workunit.client.0.smithi006.stdout:checking for memory.h... yes 2024-02-26T06:29:42.502 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strings.h... yes 2024-02-26T06:29:42.599 INFO:tasks.workunit.client.0.smithi006.stdout:checking for inttypes.h... yes 2024-02-26T06:29:42.693 INFO:tasks.workunit.client.0.smithi006.stdout:checking for stdint.h... yes 2024-02-26T06:29:42.782 INFO:tasks.workunit.client.0.smithi006.stdout:checking for unistd.h... yes 2024-02-26T06:29:42.887 INFO:tasks.workunit.client.0.smithi006.stdout:checking for dlfcn.h... yes 2024-02-26T06:29:42.905 INFO:tasks.workunit.client.0.smithi006.stdout:checking for objdir... .libs 2024-02-26T06:29:43.171 INFO:tasks.workunit.client.0.smithi006.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-02-26T06:29:43.171 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-02-26T06:29:43.259 INFO:tasks.workunit.client.0.smithi006.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-02-26T06:29:43.348 INFO:tasks.workunit.client.0.smithi006.stdout:checking if gcc static flag -static works... no 2024-02-26T06:29:43.488 INFO:tasks.workunit.client.0.smithi006.stdout:checking if gcc supports -c -o file.o... yes 2024-02-26T06:29:43.489 INFO:tasks.workunit.client.0.smithi006.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-02-26T06:29:43.516 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-02-26T06:29:43.639 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether -lc should be explicitly linked in... no 2024-02-26T06:29:43.667 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:43 smithi090 ceph-mon[35681]: pgmap v51: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 494 op/s 2024-02-26T06:29:43.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:43 smithi006 ceph-mon[31790]: pgmap v51: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 494 op/s 2024-02-26T06:29:43.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:43 smithi167 ceph-mon[36229]: pgmap v51: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 494 op/s 2024-02-26T06:29:43.802 INFO:tasks.workunit.client.0.smithi006.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-02-26T06:29:43.802 INFO:tasks.workunit.client.0.smithi006.stdout:checking how to hardcode library paths into programs... immediate 2024-02-26T06:29:43.807 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether stripping libraries is possible... yes 2024-02-26T06:29:43.807 INFO:tasks.workunit.client.0.smithi006.stdout:checking if libtool supports shared libraries... yes 2024-02-26T06:29:43.808 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether to build shared libraries... yes 2024-02-26T06:29:43.808 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether to build static libraries... yes 2024-02-26T06:29:43.832 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc... (cached) gcc 2024-02-26T06:29:43.915 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-02-26T06:29:43.915 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc accepts -g... (cached) yes 2024-02-26T06:29:43.917 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-02-26T06:29:43.918 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gmake... /usr/bin/gmake 2024-02-26T06:29:43.919 INFO:tasks.workunit.client.0.smithi006.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-02-26T06:29:43.919 INFO:tasks.workunit.client.0.smithi006.stdout:checking for tar... /bin/tar 2024-02-26T06:29:43.920 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gzip... /bin/gzip 2024-02-26T06:29:43.966 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether gcc -MM is supported... yes 2024-02-26T06:29:43.967 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sort... /bin/sort 2024-02-26T06:29:43.967 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether ln -s works... yes 2024-02-26T06:29:43.968 INFO:tasks.workunit.client.0.smithi006.stdout:checking for rpm... /bin/rpm 2024-02-26T06:29:43.982 INFO:tasks.workunit.client.0.smithi006.stdout:checking for rpmbuild... no 2024-02-26T06:29:43.982 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ANSI C header files... (cached) yes 2024-02-26T06:29:44.090 INFO:tasks.workunit.client.0.smithi006.stdout:checking assert.h usability... yes 2024-02-26T06:29:44.127 INFO:tasks.workunit.client.0.smithi006.stdout:checking assert.h presence... yes 2024-02-26T06:29:44.127 INFO:tasks.workunit.client.0.smithi006.stdout:checking for assert.h... yes 2024-02-26T06:29:44.242 INFO:tasks.workunit.client.0.smithi006.stdout:checking bstring.h usability... no 2024-02-26T06:29:44.306 INFO:tasks.workunit.client.0.smithi006.stdout:checking bstring.h presence... no 2024-02-26T06:29:44.306 INFO:tasks.workunit.client.0.smithi006.stdout:checking for bstring.h... no 2024-02-26T06:29:44.399 INFO:tasks.workunit.client.0.smithi006.stdout:checking libgen.h usability... yes 2024-02-26T06:29:44.439 INFO:tasks.workunit.client.0.smithi006.stdout:checking libgen.h presence... yes 2024-02-26T06:29:44.439 INFO:tasks.workunit.client.0.smithi006.stdout:checking for libgen.h... yes 2024-02-26T06:29:44.566 INFO:tasks.workunit.client.0.smithi006.stdout:checking dirent.h usability... yes 2024-02-26T06:29:44.607 INFO:tasks.workunit.client.0.smithi006.stdout:checking dirent.h presence... yes 2024-02-26T06:29:44.608 INFO:tasks.workunit.client.0.smithi006.stdout:checking for dirent.h... yes 2024-02-26T06:29:44.706 INFO:tasks.workunit.client.0.smithi006.stdout:checking errno.h usability... yes 2024-02-26T06:29:44.744 INFO:tasks.workunit.client.0.smithi006.stdout:checking errno.h presence... yes 2024-02-26T06:29:44.744 INFO:tasks.workunit.client.0.smithi006.stdout:checking for errno.h... yes 2024-02-26T06:29:44.866 INFO:tasks.workunit.client.0.smithi006.stdout:checking malloc.h usability... yes 2024-02-26T06:29:44.909 INFO:tasks.workunit.client.0.smithi006.stdout:checking malloc.h presence... yes 2024-02-26T06:29:44.909 INFO:tasks.workunit.client.0.smithi006.stdout:checking for malloc.h... yes 2024-02-26T06:29:45.027 INFO:tasks.workunit.client.0.smithi006.stdout:checking uuid.h usability... no 2024-02-26T06:29:45.093 INFO:tasks.workunit.client.0.smithi006.stdout:checking uuid.h presence... no 2024-02-26T06:29:45.093 INFO:tasks.workunit.client.0.smithi006.stdout:checking for uuid.h... no 2024-02-26T06:29:45.211 INFO:tasks.workunit.client.0.smithi006.stdout:checking uuid/uuid.h usability... yes 2024-02-26T06:29:45.256 INFO:tasks.workunit.client.0.smithi006.stdout:checking uuid/uuid.h presence... yes 2024-02-26T06:29:45.256 INFO:tasks.workunit.client.0.smithi006.stdout:checking for uuid/uuid.h... yes 2024-02-26T06:29:45.373 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/uuid.h usability... no 2024-02-26T06:29:45.442 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/uuid.h presence... no 2024-02-26T06:29:45.442 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/uuid.h... no 2024-02-26T06:29:45.547 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/file.h usability... yes 2024-02-26T06:29:45.583 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/file.h presence... yes 2024-02-26T06:29:45.583 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/file.h... yes 2024-02-26T06:29:45.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:45 smithi006 ceph-mon[31790]: pgmap v52: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1005 KiB/s wr, 172 op/s 2024-02-26T06:29:45.702 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/fcntl.h usability... yes 2024-02-26T06:29:45.741 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/fcntl.h presence... yes 2024-02-26T06:29:45.741 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/fcntl.h... yes 2024-02-26T06:29:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:45 smithi167 ceph-mon[36229]: pgmap v52: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1005 KiB/s wr, 172 op/s 2024-02-26T06:29:45.855 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/syssgi.h usability... no 2024-02-26T06:29:45.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:45 smithi090 ceph-mon[35681]: pgmap v52: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1005 KiB/s wr, 172 op/s 2024-02-26T06:29:45.919 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/syssgi.h presence... no 2024-02-26T06:29:45.919 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/syssgi.h... no 2024-02-26T06:29:46.097 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/param.h usability... yes 2024-02-26T06:29:46.147 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/param.h presence... yes 2024-02-26T06:29:46.147 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/param.h... yes 2024-02-26T06:29:46.180 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/stat.h... (cached) yes 2024-02-26T06:29:46.279 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/statvfs.h usability... yes 2024-02-26T06:29:46.322 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/statvfs.h presence... yes 2024-02-26T06:29:46.322 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/statvfs.h... yes 2024-02-26T06:29:46.428 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/time.h usability... yes 2024-02-26T06:29:46.469 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/time.h presence... yes 2024-02-26T06:29:46.470 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/time.h... yes 2024-02-26T06:29:46.574 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/ioctl.h usability... yes 2024-02-26T06:29:46.610 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/ioctl.h presence... yes 2024-02-26T06:29:46.610 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/ioctl.h... yes 2024-02-26T06:29:46.730 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/wait.h usability... yes 2024-02-26T06:29:46.775 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/wait.h presence... yes 2024-02-26T06:29:46.775 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/wait.h... yes 2024-02-26T06:29:46.807 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/types.h... (cached) yes 2024-02-26T06:29:46.828 INFO:tasks.workunit.client.0.smithi006.stdout:checking for strings.h... (cached) yes 2024-02-26T06:29:46.925 INFO:tasks.workunit.client.0.smithi006.stdout:checking err.h usability... yes 2024-02-26T06:29:46.965 INFO:tasks.workunit.client.0.smithi006.stdout:checking err.h presence... yes 2024-02-26T06:29:46.965 INFO:tasks.workunit.client.0.smithi006.stdout:checking for err.h... yes 2024-02-26T06:29:47.074 INFO:tasks.workunit.client.0.smithi006.stdout:checking linux/falloc.h usability... yes 2024-02-26T06:29:47.112 INFO:tasks.workunit.client.0.smithi006.stdout:checking linux/falloc.h presence... yes 2024-02-26T06:29:47.113 INFO:tasks.workunit.client.0.smithi006.stdout:checking for linux/falloc.h... yes 2024-02-26T06:29:47.231 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-02-26T06:29:47.292 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-02-26T06:29:47.292 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/fs/xfs_fsops.h... no 2024-02-26T06:29:47.394 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/fs/xfs_itable.h usability... no 2024-02-26T06:29:47.455 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/fs/xfs_itable.h presence... no 2024-02-26T06:29:47.455 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/fs/xfs_itable.h... no 2024-02-26T06:29:47.553 INFO:tasks.workunit.client.0.smithi006.stdout:checking xfs/platform_defs.h usability... no 2024-02-26T06:29:47.613 INFO:tasks.workunit.client.0.smithi006.stdout:checking xfs/platform_defs.h presence... no 2024-02-26T06:29:47.614 INFO:tasks.workunit.client.0.smithi006.stdout:checking for xfs/platform_defs.h... no 2024-02-26T06:29:47.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:47 smithi006 ceph-mon[31790]: pgmap v53: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 582 KiB/s wr, 111 op/s 2024-02-26T06:29:47.707 INFO:tasks.workunit.client.0.smithi006.stdout:checking btrfs/ioctl.h usability... no 2024-02-26T06:29:47.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:47 smithi167 ceph-mon[36229]: pgmap v53: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 582 KiB/s wr, 111 op/s 2024-02-26T06:29:47.767 INFO:tasks.workunit.client.0.smithi006.stdout:checking btrfs/ioctl.h presence... no 2024-02-26T06:29:47.767 INFO:tasks.workunit.client.0.smithi006.stdout:checking for btrfs/ioctl.h... no 2024-02-26T06:29:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:47 smithi090 ceph-mon[35681]: pgmap v53: 97 pgs: 97 active+clean; 58 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 582 KiB/s wr, 111 op/s 2024-02-26T06:29:47.872 INFO:tasks.workunit.client.0.smithi006.stdout:checking cifs/ioctl.h usability... no 2024-02-26T06:29:47.932 INFO:tasks.workunit.client.0.smithi006.stdout:checking cifs/ioctl.h presence... no 2024-02-26T06:29:47.932 INFO:tasks.workunit.client.0.smithi006.stdout:checking for cifs/ioctl.h... no 2024-02-26T06:29:48.025 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/mman.h usability... yes 2024-02-26T06:29:48.066 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/mman.h presence... yes 2024-02-26T06:29:48.066 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/mman.h... yes 2024-02-26T06:29:48.157 INFO:tasks.workunit.client.0.smithi006.stdout:checking for xfs/xfs_log_format.h... no 2024-02-26T06:29:48.281 INFO:tasks.workunit.client.0.smithi006.stdout:checking for uuid_compare... no 2024-02-26T06:29:48.376 INFO:tasks.workunit.client.0.smithi006.stdout:checking for uuid_compare in -luuid... yes 2024-02-26T06:29:48.524 INFO:tasks.workunit.client.0.smithi006.stdout:checking for xfs/xfs.h... yes 2024-02-26T06:29:48.595 INFO:tasks.workunit.client.0.smithi006.stdout:checking for xfs/libxfs.h... no 2024-02-26T06:29:48.672 INFO:tasks.workunit.client.0.smithi006.stdout:checking whether xlog_assign_lsn is declared... no 2024-02-26T06:29:48.761 INFO:tasks.workunit.client.0.smithi006.stdout:checking for xfs/xqm.h... yes 2024-02-26T06:29:48.881 INFO:tasks.workunit.client.0.smithi006.stdout:checking xfsctl from xfs/xfs.h... ok 2024-02-26T06:29:48.959 INFO:tasks.workunit.client.0.smithi006.stdout:checking xfs/handle.h usability... yes 2024-02-26T06:29:48.996 INFO:tasks.workunit.client.0.smithi006.stdout:checking xfs/handle.h presence... yes 2024-02-26T06:29:48.996 INFO:tasks.workunit.client.0.smithi006.stdout:checking for xfs/handle.h... yes 2024-02-26T06:29:49.098 INFO:tasks.workunit.client.0.smithi006.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-02-26T06:29:49.230 INFO:tasks.workunit.client.0.smithi006.stdout:checking attr/attributes.h usability... yes 2024-02-26T06:29:49.268 INFO:tasks.workunit.client.0.smithi006.stdout:checking attr/attributes.h presence... yes 2024-02-26T06:29:49.268 INFO:tasks.workunit.client.0.smithi006.stdout:checking for attr/attributes.h... yes 2024-02-26T06:29:49.378 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/attributes.h usability... no 2024-02-26T06:29:49.440 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/attributes.h presence... no 2024-02-26T06:29:49.440 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/attributes.h... no 2024-02-26T06:29:49.538 INFO:tasks.workunit.client.0.smithi006.stdout:checking for attr_list in -lattr... yes 2024-02-26T06:29:49.645 INFO:tasks.workunit.client.0.smithi006.stdout:checking for attr_set in -lattr... yes 2024-02-26T06:29:49.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:49 smithi006 ceph-mon[31790]: pgmap v54: 97 pgs: 97 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 724 KiB/s wr, 123 op/s 2024-02-26T06:29:49.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:49 smithi167 ceph-mon[36229]: pgmap v54: 97 pgs: 97 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 724 KiB/s wr, 123 op/s 2024-02-26T06:29:49.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:49 smithi090 ceph-mon[35681]: pgmap v54: 97 pgs: 97 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 724 KiB/s wr, 123 op/s 2024-02-26T06:29:50.645 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/acl.h usability... yes 2024-02-26T06:29:50.689 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/acl.h presence... yes 2024-02-26T06:29:50.689 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/acl.h... yes 2024-02-26T06:29:50.770 INFO:tasks.workunit.client.0.smithi006.stdout:checking acl/libacl.h usability... yes 2024-02-26T06:29:50.818 INFO:tasks.workunit.client.0.smithi006.stdout:checking acl/libacl.h presence... yes 2024-02-26T06:29:50.818 INFO:tasks.workunit.client.0.smithi006.stdout:checking for acl/libacl.h... yes 2024-02-26T06:29:50.924 INFO:tasks.workunit.client.0.smithi006.stdout:checking for acl_init in -lacl... yes 2024-02-26T06:29:51.030 INFO:tasks.workunit.client.0.smithi006.stdout:checking gdbm-ndbm.h usability... no 2024-02-26T06:29:51.091 INFO:tasks.workunit.client.0.smithi006.stdout:checking gdbm-ndbm.h presence... no 2024-02-26T06:29:51.091 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gdbm-ndbm.h... no 2024-02-26T06:29:51.175 INFO:tasks.workunit.client.0.smithi006.stdout:checking gdbm/ndbm.h usability... no 2024-02-26T06:29:51.239 INFO:tasks.workunit.client.0.smithi006.stdout:checking gdbm/ndbm.h presence... no 2024-02-26T06:29:51.239 INFO:tasks.workunit.client.0.smithi006.stdout:checking for gdbm/ndbm.h... no 2024-02-26T06:29:51.321 INFO:tasks.workunit.client.0.smithi006.stdout:checking ndbm.h usability... no 2024-02-26T06:29:51.388 INFO:tasks.workunit.client.0.smithi006.stdout:checking ndbm.h presence... no 2024-02-26T06:29:51.388 INFO:tasks.workunit.client.0.smithi006.stdout:checking for ndbm.h... no 2024-02-26T06:29:51.472 INFO:tasks.workunit.client.0.smithi006.stdout:checking libaio.h usability... yes 2024-02-26T06:29:51.516 INFO:tasks.workunit.client.0.smithi006.stdout:checking libaio.h presence... yes 2024-02-26T06:29:51.517 INFO:tasks.workunit.client.0.smithi006.stdout:checking for libaio.h... yes 2024-02-26T06:29:51.631 INFO:tasks.workunit.client.0.smithi006.stdout:checking for dm_make_handle in -ldm... no 2024-02-26T06:29:51.632 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-02-26T06:29:51.632 INFO:tasks.workunit.client.0.smithi006.stdout:WARNING: could not find a valid DMAPI base library. 2024-02-26T06:29:51.632 INFO:tasks.workunit.client.0.smithi006.stdout:If you want DMAPI support please install the data migration 2024-02-26T06:29:51.632 INFO:tasks.workunit.client.0.smithi006.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-02-26T06:29:51.632 INFO:tasks.workunit.client.0.smithi006.stdout:from the dmapi source. 2024-02-26T06:29:51.632 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-02-26T06:29:51.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:51 smithi006 ceph-mon[31790]: pgmap v55: 97 pgs: 97 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 230 KiB/s wr, 37 op/s 2024-02-26T06:29:51.709 INFO:tasks.workunit.client.0.smithi006.stdout:checking linux/fiemap.h usability... yes 2024-02-26T06:29:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:51 smithi167 ceph-mon[36229]: pgmap v55: 97 pgs: 97 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 230 KiB/s wr, 37 op/s 2024-02-26T06:29:51.749 INFO:tasks.workunit.client.0.smithi006.stdout:checking linux/fiemap.h presence... yes 2024-02-26T06:29:51.749 INFO:tasks.workunit.client.0.smithi006.stdout:checking for linux/fiemap.h... yes 2024-02-26T06:29:51.846 INFO:tasks.workunit.client.0.smithi006.stdout:checking for fallocate... yes 2024-02-26T06:29:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:51 smithi090 ceph-mon[35681]: pgmap v55: 97 pgs: 97 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 230 KiB/s wr, 37 op/s 2024-02-26T06:29:51.949 INFO:tasks.workunit.client.0.smithi006.stdout:checking for open_by_handle_at... yes 2024-02-26T06:29:52.048 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/prctl.h usability... yes 2024-02-26T06:29:52.088 INFO:tasks.workunit.client.0.smithi006.stdout:checking sys/prctl.h presence... yes 2024-02-26T06:29:52.089 INFO:tasks.workunit.client.0.smithi006.stdout:checking for sys/prctl.h... yes 2024-02-26T06:29:52.170 INFO:tasks.workunit.client.0.smithi006.stdout:checking linux/fs.h usability... yes 2024-02-26T06:29:52.207 INFO:tasks.workunit.client.0.smithi006.stdout:checking linux/fs.h presence... yes 2024-02-26T06:29:52.208 INFO:tasks.workunit.client.0.smithi006.stdout:checking for linux/fs.h... yes 2024-02-26T06:29:52.315 INFO:tasks.workunit.client.0.smithi006.stdout:checking btrfsutil.h usability... no 2024-02-26T06:29:52.381 INFO:tasks.workunit.client.0.smithi006.stdout:checking btrfsutil.h presence... no 2024-02-26T06:29:52.381 INFO:tasks.workunit.client.0.smithi006.stdout:checking for btrfsutil.h... no 2024-02-26T06:29:52.505 INFO:tasks.workunit.client.0.smithi006.stdout:checking for copy_file_range... yes 2024-02-26T06:29:52.613 INFO:tasks.workunit.client.0.smithi006.stdout:checking for renameat2... yes 2024-02-26T06:29:52.662 INFO:tasks.workunit.client.0.smithi006.stdout:configure: creating ./config.status 2024-02-26T06:29:53.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:53 smithi167 ceph-mon[36229]: pgmap v56: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-02-26T06:29:53.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:53 smithi090 ceph-mon[35681]: pgmap v56: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-02-26T06:29:53.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:53 smithi006 ceph-mon[31790]: pgmap v56: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-02-26T06:29:54.094 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating include/builddefs 2024-02-26T06:29:54.144 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: creating include/config.h 2024-02-26T06:29:54.184 INFO:tasks.workunit.client.0.smithi006.stdout:config.status: executing libtool commands 2024-02-26T06:29:54.378 INFO:tasks.workunit.client.0.smithi006.stdout:Building include 2024-02-26T06:29:54.378 INFO:tasks.workunit.client.0.smithi006.stdout:Building lib 2024-02-26T06:29:54.379 INFO:tasks.workunit.client.0.smithi006.stdout:Building tests 2024-02-26T06:29:54.390 INFO:tasks.workunit.client.0.smithi006.stdout: [LTDEP] 2024-02-26T06:29:54.882 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] databin.lo 2024-02-26T06:29:54.883 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] dataascii.lo 2024-02-26T06:29:54.883 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] datapid.lo 2024-02-26T06:29:54.884 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] file_lock.lo 2024-02-26T06:29:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.926+0000 7f912ba00640 1 -- 172.21.15.6:0/2749012034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124108690 msgr2=0x7f9124110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.926+0000 7f912ba00640 1 --2- 172.21.15.6:0/2749012034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124108690 0x7f9124110260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f911c009be0 tx=0x7f911c02bef0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 -- 172.21.15.6:0/2749012034 shutdown_connections 2024-02-26T06:29:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 --2- 172.21.15.6:0/2749012034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124108690 0x7f9124110260 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 --2- 172.21.15.6:0/2749012034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9124103f70 0x7f9124107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 --2- 172.21.15.6:0/2749012034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91241035c0 0x7f91241039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 -- 172.21.15.6:0/2749012034 >> 172.21.15.6:0/2749012034 conn(0x7f91240fee00 msgr2=0x7f9124101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 -- 172.21.15.6:0/2749012034 shutdown_connections 2024-02-26T06:29:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.927+0000 7f912ba00640 1 -- 172.21.15.6:0/2749012034 wait complete. 2024-02-26T06:29:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.929+0000 7f912ba00640 1 Processor -- start 2024-02-26T06:29:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.929+0000 7f912ba00640 1 -- start start 2024-02-26T06:29:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f912ba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91241035c0 0x7f912419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f912ba00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9124108690 0x7f912419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f912ba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 0x7f9124199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f912ba00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912419a280 con 0x7f9124108690 2024-02-26T06:29:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f912ba00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912419a3f0 con 0x7f91241997d0 2024-02-26T06:29:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f912ba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912419a560 con 0x7f91241035c0 2024-02-26T06:29:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f9129a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 0x7f9124199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f9129a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 0x7f9124199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:54804/0 (socket says 172.21.15.6:54804) 2024-02-26T06:29:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f9129a00640 1 -- 172.21.15.6:0/289828057 learned_addr learned my addr 172.21.15.6:0/289828057 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.930+0000 7f9123e00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9124108690 0x7f912419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9129a00640 1 -- 172.21.15.6:0/289828057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91241035c0 msgr2=0x7f912419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9129a00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91241035c0 0x7f912419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9129a00640 1 -- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9124108690 msgr2=0x7f912419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9129a00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9124108690 0x7f912419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9129a00640 1 -- 172.21.15.6:0/289828057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c009840 con 0x7f91241997d0 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9129a00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 0x7f9124199c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f911c036040 tx=0x7f911c02cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.931+0000 7f9121600640 1 -- 172.21.15.6:0/289828057 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c004550 con 0x7f91241997d0 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.932+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91241a64b0 con 0x7f91241997d0 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.932+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91241a6960 con 0x7f91241997d0 2024-02-26T06:29:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.932+0000 7f9121600640 1 -- 172.21.15.6:0/289828057 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f911c0046b0 con 0x7f91241997d0 2024-02-26T06:29:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.932+0000 7f9121600640 1 -- 172.21.15.6:0/289828057 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c03d800 con 0x7f91241997d0 2024-02-26T06:29:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.938+0000 7f912ea36740 1 Processor -- start 2024-02-26T06:29:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f9121600640 1 -- 172.21.15.6:0/289828057 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f911c0344d0 con 0x7f91241997d0 2024-02-26T06:29:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 -- start start 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfb9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfb9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57df6c5f0 con 0x55c57dfc2b10 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57df1e890 con 0x55c57dfba300 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.939+0000 7f912ea36740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57df1bdc0 con 0x55c57dfb9590 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfb9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfb9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:54818/0 (socket says 172.21.15.6:54818) 2024-02-26T06:29:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 -- 172.21.15.6:0/4046604620 learned_addr learned my addr 172.21.15.6:0/4046604620 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9129000640 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfb9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 -- 172.21.15.6:0/4046604620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 msgr2=0x55c57dfb9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfb9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 -- 172.21.15.6:0/4046604620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 msgr2=0x55c57dfc4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.940+0000 7f9123e00640 1 -- 172.21.15.6:0/4046604620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c57de98ad0 con 0x55c57dfba300 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9121600640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f910c0a22a0 0x7f910c0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9121600640 1 -- 172.21.15.6:0/289828057 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f911c0f0520 con 0x7f91241997d0 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9123e00640 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfb9050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9110009c20 tx=0x7f9110011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9492657cdc42b113 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9129000640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f910c0a22a0 0x7f910c0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9116000640 1 -- 172.21.15.6:0/4046604620 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110016070 con 0x55c57dfba300 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9116000640 1 -- 172.21.15.6:0/4046604620 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9110009d10 con 0x55c57dfba300 2024-02-26T06:29:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.941+0000 7f9116000640 1 -- 172.21.15.6:0/4046604620 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911001a6b0 con 0x55c57dfba300 2024-02-26T06:29:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 -- 172.21.15.6:0/4046604620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 msgr2=0x55c57dfb9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfb9050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9110009c20 tx=0x7f9110011810 comp rx=0 tx=0).stop 2024-02-26T06:29:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f9129000640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f910c0a22a0 0x7f910c0a4760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91180096f0 tx=0x7f9118009680 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 -- 172.21.15.6:0/4046604620 shutdown_connections 2024-02-26T06:29:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfb9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 --2- 172.21.15.6:0/4046604620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfb9050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.943+0000 7f912ea36740 1 -- 172.21.15.6:0/4046604620 >> 172.21.15.6:0/4046604620 conn(0x55c57df78480 msgr2=0x55c57dfb4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.944+0000 7f912ea36740 1 -- 172.21.15.6:0/4046604620 shutdown_connections 2024-02-26T06:29:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.944+0000 7f912ea36740 1 -- 172.21.15.6:0/4046604620 wait complete. 2024-02-26T06:29:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.945+0000 7f912ea36740 1 -- start start 2024-02-26T06:29:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.945+0000 7f912ea36740 1 Processor -- start 2024-02-26T06:29:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfb9590 0x55c57dfbfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 0x55c57dfc00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57df175a0 con 0x55c57dfc2b10 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57debcb20 con 0x55c57dfb9590 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57debc7c0 con 0x55c57dfba300 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f912ea36740 10 client.0 _ll_register_callbacks cb 0x55c57ded0520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.946+0000 7f9123e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 0x55c57dfc00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9129000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfb9590 0x55c57dfbfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9123e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 0x55c57dfc00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:55086/0 (socket says 172.21.15.6:55086) 2024-02-26T06:29:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9123e00640 1 -- 172.21.15.6:0/3808473235 learned_addr learned my addr 172.21.15.6:0/3808473235 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9123e00640 1 -- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfb9590 msgr2=0x55c57dfbfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9123e00640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfb9590 0x55c57dfbfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9123e00640 1 -- 172.21.15.6:0/3808473235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 msgr2=0x55c57dfc05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.947+0000 7f9129a00640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.948+0000 7f9123e00640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.948+0000 7f9123e00640 1 -- 172.21.15.6:0/3808473235 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9110009900 con 0x55c57dfba300 2024-02-26T06:29:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.948+0000 7f9129a00640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.949+0000 7f9123e00640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 0x55c57dfc00a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f91100058c0 tx=0x7f9110005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa0efb1ad2ed36d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.949+0000 7f910aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.949+0000 7f910aa00640 1 -- 172.21.15.6:0/3808473235 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110016070 con 0x55c57dfba300 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.949+0000 7f910aa00640 1 -- 172.21.15.6:0/3808473235 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f9110005610 con 0x55c57dfba300 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.949+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c57dee9ca0 con 0x55c57dfba300 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.950+0000 7f910aa00640 1 -- 172.21.15.6:0/3808473235 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91100048d0 con 0x55c57dfba300 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.950+0000 7f912ea36740 10 client.26050 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.950+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c57df15c20 con 0x55c57dfba300 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.951+0000 7f910aa00640 1 -- 172.21.15.6:0/3808473235 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c57dee9ca0 con 0x55c57dfba300 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.951+0000 7f910aa00640 1 -- 172.21.15.6:0/3808473235 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f911002b070 con 0x55c57dfba300 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f912ea36740 10 client.26050 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f912ea36740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f912ea36740 20 client.26050 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f912ea36740 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57dfd2a60 0x55c57dfd4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f912ea36740 4 client.26050 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:29:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c57df0dcb0 con 0x55c57dfd2a60 2024-02-26T06:29:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.952+0000 7f9129000640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57dfd2a60 0x55c57dfd4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.953+0000 7f9129000640 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57dfd2a60 0x55c57dfd4e80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21a3daf6929671c server_cookie=1d123dfcbcfec19b in_seq=0 out_seq=0 2024-02-26T06:29:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.953+0000 7f910aa00640 10 client.26050 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.954+0000 7f910aa00640 1 -- 172.21.15.6:0/3808473235 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c57df0dcb0 con 0x55c57dfd2a60 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.954+0000 7f910aa00640 10 client.26050 handle_command_reply: tid=0 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.954+0000 7f912ea36740 1 client.26050 shutdown 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.954+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 msgr2=0x55c57dfc00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.954+0000 7f912ea36740 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 0x55c57dfc00a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f91100058c0 tx=0x7f9110005420 comp rx=0 tx=0).stop 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 shutdown_connections 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57dfd2a60 0x55c57dfd4e80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfba300 0x55c57dfc00a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 --2- 172.21.15.6:0/3808473235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfb9590 0x55c57dfbfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 >> 172.21.15.6:0/3808473235 conn(0x55c57df78480 msgr2=0x55c57dfb5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f910aa00640 0 client.26050 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 shutdown_connections 2024-02-26T06:29:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.955+0000 7f912ea36740 1 -- 172.21.15.6:0/3808473235 wait complete. 2024-02-26T06:29:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.956+0000 7f912ea36740 20 client.26050 trim_cache size 0 max 16384 2024-02-26T06:29:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 Processor -- start 2024-02-26T06:29:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 -- start start 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfc9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57dc5fe60 con 0x55c57dfc2b10 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57df0dce0 con 0x55c57dfba300 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.978+0000 7f912ea36740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57dfcf630 con 0x55c57dfb9590 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:55102/0 (socket says 172.21.15.6:55102) 2024-02-26T06:29:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129000640 1 -- 172.21.15.6:0/1628851141 learned_addr learned my addr 172.21.15.6:0/1628851141 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9123e00640 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfc9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129a00640 1 -- 172.21.15.6:0/1628851141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 msgr2=0x55c57dfc0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129a00640 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129a00640 1 -- 172.21.15.6:0/1628851141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 msgr2=0x55c57dfc9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129a00640 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfc9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9129a00640 1 -- 172.21.15.6:0/1628851141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c57dfbb7a0 con 0x55c57dfc2b10 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.979+0000 7f9123e00640 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfc9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.980+0000 7f9129a00640 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f911c0edd00 tx=0x7f911c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6219ca33902be1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.980+0000 7f9115600640 1 -- 172.21.15.6:0/1628851141 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0c5740 con 0x55c57dfc2b10 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.980+0000 7f9115600640 1 -- 172.21.15.6:0/1628851141 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f911c0d5070 con 0x55c57dfc2b10 2024-02-26T06:29:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.980+0000 7f9115600640 1 -- 172.21.15.6:0/1628851141 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c004d40 con 0x55c57dfc2b10 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.981+0000 7f912ea36740 1 -- 172.21.15.6:0/1628851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 msgr2=0x55c57dfc9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.981+0000 7f912ea36740 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f911c0edd00 tx=0x7f911c004960 comp rx=0 tx=0).stop 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 -- 172.21.15.6:0/1628851141 shutdown_connections 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9e80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfba300 0x55c57dfc9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 --2- 172.21.15.6:0/1628851141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 -- 172.21.15.6:0/1628851141 >> 172.21.15.6:0/1628851141 conn(0x55c57df78480 msgr2=0x55c57dfbe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 -- 172.21.15.6:0/1628851141 shutdown_connections 2024-02-26T06:29:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.982+0000 7f912ea36740 1 -- 172.21.15.6:0/1628851141 wait complete. 2024-02-26T06:29:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.983+0000 7f912ea36740 1 -- start start 2024-02-26T06:29:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.983+0000 7f912ea36740 1 Processor -- start 2024-02-26T06:29:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfba300 0x55c57dfc8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57e029060 con 0x55c57dfba300 2024-02-26T06:29:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57e0291d0 con 0x55c57dfc2b10 2024-02-26T06:29:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57e029390 con 0x55c57dfb9590 2024-02-26T06:29:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f912ea36740 10 client.0 _ll_register_callbacks cb 0x7f9110009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f9129000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f9129000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:55114/0 (socket says 172.21.15.6:55114) 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f9129000640 1 -- 172.21.15.6:0/2035659903 learned_addr learned my addr 172.21.15.6:0/2035659903 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.984+0000 7f9123e00640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfba300 0x55c57dfc8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129a00640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129000640 1 -- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfc2b10 msgr2=0x55c57dfc9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129000640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129000640 1 -- 172.21.15.6:0/2035659903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfba300 msgr2=0x55c57dfc8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129000640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfba300 0x55c57dfc8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129000640 1 -- 172.21.15.6:0/2035659903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c57df15c20 con 0x55c57dfb9590 2024-02-26T06:29:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9123e00640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfba300 0x55c57dfc8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:29:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f9129000640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc85f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9118004250 tx=0x7f91180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783f565dab82d4c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:29:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.985+0000 7f910aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-02-26T06:29:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.986+0000 7f910aa00640 1 -- 172.21.15.6:0/2035659903 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118026740 con 0x55c57dfb9590 2024-02-26T06:29:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.986+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c57df3cd00 con 0x55c57dfb9590 2024-02-26T06:29:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.986+0000 7f910aa00640 1 -- 172.21.15.6:0/2035659903 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f91180268a0 con 0x55c57dfb9590 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.986+0000 7f910aa00640 1 -- 172.21.15.6:0/2035659903 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9118003690 con 0x55c57dfb9590 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.986+0000 7f912ea36740 10 client.26056 fetch_fsmap learned FSMap version 35 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.986+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f91241a6960 con 0x55c57dfb9590 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.987+0000 7f910aa00640 1 -- 172.21.15.6:0/2035659903 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c57df3cd00 con 0x55c57dfb9590 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.987+0000 7f910aa00640 1 -- 172.21.15.6:0/2035659903 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f911801a040 con 0x55c57dfb9590 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f912ea36740 10 client.26056 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:29:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f912ea36740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f912ea36740 20 client.26056 populate_metadata read hostname 'smithi006' 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f912ea36740 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57e02ac20 0x55c57e02d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f912ea36740 4 client.26056 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c57dc17600 con 0x55c57e02ac20 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.988+0000 7f9123e00640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57e02ac20 0x55c57e02d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.989+0000 7f9123e00640 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57e02ac20 0x55c57e02d040 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6b76d5d2e2ad66 server_cookie=dc4ac3af6fc2d1af in_seq=0 out_seq=0 2024-02-26T06:29:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.989+0000 7f910aa00640 10 client.26056 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.998+0000 7f910aa00640 1 -- 172.21.15.6:0/2035659903 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c57dc17600 con 0x55c57e02ac20 2024-02-26T06:29:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f910aa00640 10 client.26056 handle_command_reply: tid=0 2024-02-26T06:29:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 client.26056 shutdown 2024-02-26T06:29:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 msgr2=0x55c57dfc85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc85f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9118004250 tx=0x7f91180046c0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 shutdown_connections 2024-02-26T06:29:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x55c57e02ac20 0x55c57e02d040 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c57dfc2b10 0x55c57dfc9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57dfba300 0x55c57dfc8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 --2- 172.21.15.6:0/2035659903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c57dfb9590 0x55c57dfc85f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 >> 172.21.15.6:0/2035659903 conn(0x55c57df78480 msgr2=0x55c57dfaa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:54.999+0000 7f910aa00640 0 client.26056 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 shutdown_connections 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ea36740 1 -- 172.21.15.6:0/2035659903 wait complete. 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ea36740 20 client.26056 trim_cache size 0 max 16384 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:29:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f910c0a22a0 msgr2=0x7f910c0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ba00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f910c0a22a0 0x7f910c0a4760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91180096f0 tx=0x7f9118009680 comp rx=0 tx=0).stop 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 msgr2=0x7f9124199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.000+0000 7f912ba00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 0x7f9124199c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f911c036040 tx=0x7f911c02cb30 comp rx=0 tx=0).stop 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 shutdown_connections 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f910c0a22a0 0x7f910c0a4760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91241997d0 0x7f9124199c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9124108690 0x7f912419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 --2- 172.21.15.6:0/289828057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91241035c0 0x7f912419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:29:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 >> 172.21.15.6:0/289828057 conn(0x7f91240fee00 msgr2=0x7f91241010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:29:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 shutdown_connections 2024-02-26T06:29:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:29:55.001+0000 7f912ba00640 1 -- 172.21.15.6:0/289828057 wait complete. 2024-02-26T06:29:55.143 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] forker.lo 2024-02-26T06:29:55.167 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] pattern.lo 2024-02-26T06:29:55.224 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] open_flags.lo 2024-02-26T06:29:55.226 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] random_range.lo 2024-02-26T06:29:55.424 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] string_to_tokens.lo 2024-02-26T06:29:55.424 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] str_to_bytes.lo 2024-02-26T06:29:55.526 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] tlibio.lo 2024-02-26T06:29:55.669 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] write_log.lo 2024-02-26T06:29:55.697 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] random.lo 2024-02-26T06:29:55.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:55 smithi167 ceph-mon[36229]: pgmap v57: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-02-26T06:29:55.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:55 smithi090 ceph-mon[35681]: pgmap v57: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-02-26T06:29:55.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:55 smithi006 ceph-mon[31790]: pgmap v57: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-02-26T06:29:56.044 INFO:tasks.workunit.client.0.smithi006.stdout: [LD] 2024-02-26T06:29:56.325 INFO:tasks.workunit.client.0.smithi006.stdout:Building ltp 2024-02-26T06:29:56.325 INFO:tasks.workunit.client.0.smithi006.stdout:Building src 2024-02-26T06:29:56.326 INFO:tasks.workunit.client.0.smithi006.stdout:Building m4 2024-02-26T06:29:56.327 INFO:tasks.workunit.client.0.smithi006.stdout:Building common 2024-02-26T06:29:56.344 INFO:tasks.workunit.client.0.smithi006.stdout: [DEP] 2024-02-26T06:29:56.364 INFO:tasks.workunit.client.0.smithi006.stdout: [DEP] 2024-02-26T06:29:56.781 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] doio 2024-02-26T06:29:56.782 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] fsstress 2024-02-26T06:29:56.782 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] fsx 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c: In function 'attr_list_path': 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~ 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr:In file included from ../src/global.h:25, 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr: from fsstress.c:12: 2024-02-26T06:29:57.085 INFO:tasks.workunit.client.0.smithi006.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~ 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c: In function 'attr_remove_path': 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~ 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr:In file included from ../src/global.h:25, 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: from fsstress.c:12: 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~ 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c: In function 'attr_set_path': 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-02-26T06:29:57.086 INFO:tasks.workunit.client.0.smithi006.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-02-26T06:29:57.087 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~ 2024-02-26T06:29:57.087 INFO:tasks.workunit.client.0.smithi006.stderr:In file included from ../src/global.h:25, 2024-02-26T06:29:57.087 INFO:tasks.workunit.client.0.smithi006.stderr: from fsstress.c:12: 2024-02-26T06:29:57.087 INFO:tasks.workunit.client.0.smithi006.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-02-26T06:29:57.087 INFO:tasks.workunit.client.0.smithi006.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-02-26T06:29:57.087 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~ 2024-02-26T06:29:57.098 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c: In function 'allocsp_f': 2024-02-26T06:29:57.098 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-02-26T06:29:57.098 INFO:tasks.workunit.client.0.smithi006.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-02-26T06:29:57.098 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~ 2024-02-26T06:29:57.098 INFO:tasks.workunit.client.0.smithi006.stderr: | XFS_IOC_RESVSP64 2024-02-26T06:29:57.098 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-02-26T06:29:57.118 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c: In function 'freesp_f': 2024-02-26T06:29:57.118 INFO:tasks.workunit.client.0.smithi006.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-02-26T06:29:57.118 INFO:tasks.workunit.client.0.smithi006.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-02-26T06:29:57.118 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~ 2024-02-26T06:29:57.118 INFO:tasks.workunit.client.0.smithi006.stderr: | XFS_IOC_RESVSP64 2024-02-26T06:29:57.163 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-02-26T06:29:57.163 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-02-26T06:29:57.549 INFO:tasks.workunit.client.0.smithi006.stderr:doio.c: In function 'parse_cmdline': 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr: 3113 | fprintf(stderr, 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~ 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr: 3115 | Prog, TagName, tok); 2024-02-26T06:29:57.550 INFO:tasks.workunit.client.0.smithi006.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr:fsx.c: In function 'main': 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-02-26T06:29:57.583 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-26T06:29:57.729 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-02-26T06:29:57.729 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-02-26T06:29:57.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:57 smithi167 ceph-mon[36229]: pgmap v58: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-02-26T06:29:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:57 smithi090 ceph-mon[35681]: pgmap v58: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-02-26T06:29:57.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:57 smithi006 ceph-mon[31790]: pgmap v58: 97 pgs: 97 active+clean; 73 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-02-26T06:29:59.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:29:59 smithi167 ceph-mon[36229]: pgmap v59: 97 pgs: 97 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-02-26T06:29:59.777 INFO:tasks.workunit.client.0.smithi006.stdout: [DEP] 2024-02-26T06:29:59.777 INFO:tasks.workunit.client.0.smithi006.stdout: [DEP] 2024-02-26T06:29:59.778 INFO:tasks.workunit.client.0.smithi006.stdout: [DEP] 2024-02-26T06:29:59.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:29:59 smithi090 ceph-mon[35681]: pgmap v59: 97 pgs: 97 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-02-26T06:29:59.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:29:59 smithi006 ceph-mon[31790]: pgmap v59: 97 pgs: 97 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-02-26T06:30:00.382 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] dirstress 2024-02-26T06:30:00.387 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] fill 2024-02-26T06:30:00.388 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] fill2 2024-02-26T06:30:00.389 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] getpagesize 2024-02-26T06:30:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:29:59.999+0000 7efe1ac00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:29:59.999+0000 7efe1ac00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:29:59.999+0000 7efe1ac00700 -1 log_channel(cluster) log [ERR] : mds.i(mds.2): Metadata damage detected 2024-02-26T06:30:00.675 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] holes 2024-02-26T06:30:00.750 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:00 smithi167 ceph-mon[36229]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.750 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:00 smithi167 ceph-mon[36229]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.750 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:00 smithi167 ceph-mon[36229]: mds.i(mds.2): Metadata damage detected 2024-02-26T06:30:00.750 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] lstat64 2024-02-26T06:30:00.813 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] nametest 2024-02-26T06:30:00.813 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] permname 2024-02-26T06:30:00.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:00 smithi090 ceph-mon[35681]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:00 smithi090 ceph-mon[35681]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:00 smithi090 ceph-mon[35681]: mds.i(mds.2): Metadata damage detected 2024-02-26T06:30:00.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:00 smithi006 ceph-mon[31790]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-26T06:30:00.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:00 smithi006 ceph-mon[31790]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-26T06:30:00.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:00 smithi006 ceph-mon[31790]: mds.i(mds.2): Metadata damage detected 2024-02-26T06:30:01.044 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] randholes 2024-02-26T06:30:01.061 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] runas 2024-02-26T06:30:01.178 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] truncfile 2024-02-26T06:30:01.253 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] usemem 2024-02-26T06:30:01.408 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] mmapcat 2024-02-26T06:30:01.468 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] append_reader 2024-02-26T06:30:01.512 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] append_writer 2024-02-26T06:30:01.559 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] dirperf 2024-02-26T06:30:01.707 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] metaperf 2024-02-26T06:30:01.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:01 smithi167 ceph-mon[36229]: pgmap v60: 97 pgs: 97 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-02-26T06:30:01.753 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] devzero 2024-02-26T06:30:01.809 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] feature 2024-02-26T06:30:01.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:01 smithi090 ceph-mon[35681]: pgmap v60: 97 pgs: 97 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-02-26T06:30:01.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:01 smithi006 ceph-mon[31790]: pgmap v60: 97 pgs: 97 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-02-26T06:30:01.947 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] alloc 2024-02-26T06:30:02.070 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] fault 2024-02-26T06:30:02.172 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] fstest 2024-02-26T06:30:02.173 INFO:tasks.workunit.client.0.smithi006.stdout: [CC] t_access_root 2024-02-26T06:30:02.219 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c: In function 'main': 2024-02-26T06:30:02.219 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: 154 | bozo! 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~ 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: 154 | bozo! 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: | ; 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: 184 | rflag = 1; 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~ 2024-02-26T06:30:02.220 INFO:tasks.workunit.client.0.smithi006.stderr: | nflag 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr: 320 | bozo! 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr: | ; 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr: 128 | struct flock64 f; 2024-02-26T06:30:02.221 INFO:tasks.workunit.client.0.smithi006.stderr: | ^ 2024-02-26T06:30:02.227 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-02-26T06:30:02.227 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-02-26T06:30:02.453 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c: In function 'run_child': 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~ 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~ 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-26T06:30:02.454 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~ 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~ 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-26T06:30:02.455 INFO:tasks.workunit.client.0.smithi006.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-26T06:30:02.562 INFO:tasks.workunit.client.0.smithi006.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-02-26T06:30:02.562 INFO:tasks.workunit.client.0.smithi006.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-02-26T06:30:02.563 INFO:tasks.workunit.client.0.smithi006.stderr:make: *** [Makefile:51: default] Error 2 2024-02-26T06:30:02.564 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-02-26T06:30:02.565 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-02-26T06:30:02.565 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-26T06:30:03.645 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi006 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-02-26T06:30:08.824 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=948a37067c4e4025b3c8fa9ef7475912 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi006 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-02-26T06:30:08.827 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-02-26T06:30:08.836 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:30:08.842 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:03 smithi090 ceph-mon[35681]: pgmap v61: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.5 MiB/s wr, 242 op/s 2024-02-26T06:30:08.843 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:05 smithi090 ceph-mon[35681]: pgmap v62: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 187 KiB/s wr, 63 op/s 2024-02-26T06:30:08.843 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:07 smithi090 ceph-mon[35681]: pgmap v63: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 187 KiB/s wr, 63 op/s 2024-02-26T06:30:08.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:03 smithi167 ceph-mon[36229]: pgmap v61: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.5 MiB/s wr, 242 op/s 2024-02-26T06:30:08.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:05 smithi167 ceph-mon[36229]: pgmap v62: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 187 KiB/s wr, 63 op/s 2024-02-26T06:30:08.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:07 smithi167 ceph-mon[36229]: pgmap v63: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 187 KiB/s wr, 63 op/s 2024-02-26T06:30:08.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:03 smithi006 ceph-mon[31790]: pgmap v61: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.5 MiB/s wr, 242 op/s 2024-02-26T06:30:08.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:05 smithi006 ceph-mon[31790]: pgmap v62: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 187 KiB/s wr, 63 op/s 2024-02-26T06:30:08.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:07 smithi006 ceph-mon[31790]: pgmap v63: 97 pgs: 97 active+clean; 74 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 187 KiB/s wr, 63 op/s 2024-02-26T06:30:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 -- 172.21.15.6:0/1498714890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7700100d20 msgr2=0x7f7700104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 --2- 172.21.15.6:0/1498714890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7700100d20 0x7f7700104b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009900 tx=0x7f76ec02bc30 comp rx=0 tx=0).stop 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 -- 172.21.15.6:0/1498714890 shutdown_connections 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 --2- 172.21.15.6:0/1498714890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700105240 0x7f770010cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 --2- 172.21.15.6:0/1498714890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7700100d20 0x7f7700104b10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 --2- 172.21.15.6:0/1498714890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7700100370 0x7f7700100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.150+0000 7f7707000640 1 -- 172.21.15.6:0/1498714890 >> 172.21.15.6:0/1498714890 conn(0x7f77000759e0 msgr2=0x7f7700075df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.151+0000 7f7707000640 1 -- 172.21.15.6:0/1498714890 shutdown_connections 2024-02-26T06:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.151+0000 7f7707000640 1 -- 172.21.15.6:0/1498714890 wait complete. 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.152+0000 7f7707000640 1 Processor -- start 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.152+0000 7f7707000640 1 -- start start 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7707000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 0x7f7700197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7707000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7700105240 0x7f7700197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7707000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770019be50 0x7f770019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7707000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019c840 con 0x7f7700105240 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7707000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019c9b0 con 0x7f7700100370 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7707000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019cb20 con 0x7f770019be50 2024-02-26T06:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7705c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 0x7f7700197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7705c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 0x7f7700197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:40412/0 (socket says 172.21.15.6:40412) 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7705c00640 1 -- 172.21.15.6:0/1773248419 learned_addr learned my addr 172.21.15.6:0/1773248419 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7705200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7700105240 0x7f7700197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.153+0000 7f7706600640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770019be50 0x7f770019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7705c00640 1 -- 172.21.15.6:0/1773248419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770019be50 msgr2=0x7f770019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7705c00640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770019be50 0x7f770019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7705c00640 1 -- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7700105240 msgr2=0x7f7700197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7705c00640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7700105240 0x7f7700197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7705c00640 1 -- 172.21.15.6:0/1773248419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec009560 con 0x7f7700100370 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7705c00640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 0x7f7700197590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76f400ebc0 tx=0x7f76f400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f76fe000640 1 -- 172.21.15.6:0/1773248419 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40099b0 con 0x7f7700100370 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f76fe000640 1 -- 172.21.15.6:0/1773248419 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f400eec0 con 0x7f7700100370 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f76fe000640 1 -- 172.21.15.6:0/1773248419 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4018b60 con 0x7f7700100370 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.154+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001a4100 con 0x7f7700100370 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.155+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a4530 con 0x7f7700100370 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 Processor -- start 2024-02-26T06:30:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 -- start start 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f76fe000640 1 -- 172.21.15.6:0/1773248419 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f76f4018cc0 con 0x7f7700100370 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b8140 0x559f6c5b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 0x559f6c5c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c56e5f0 con 0x559f6c5b73d0 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c520890 con 0x559f6c5c0950 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.158+0000 7f770b5a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c51ddc0 con 0x559f6c5b8140 2024-02-26T06:30:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 0x559f6c5c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7706600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b8140 0x559f6c5b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 0x559f6c5c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:40426/0 (socket says 172.21.15.6:40426) 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 -- 172.21.15.6:0/3216842549 learned_addr learned my addr 172.21.15.6:0/3216842549 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 -- 172.21.15.6:0/3216842549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b8140 msgr2=0x559f6c5b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b8140 0x559f6c5b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 -- 172.21.15.6:0/3216842549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 msgr2=0x559f6c5b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 -- 172.21.15.6:0/3216842549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f6c49aad0 con 0x559f6c5c0950 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705c00640 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.159+0000 7f7705200640 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 0x559f6c5c2d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009900 tx=0x7f76ec02c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb95ce75c2004a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f76f2a00640 1 -- 172.21.15.6:0/3216842549 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004800 con 0x559f6c5c0950 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f76f2a00640 1 -- 172.21.15.6:0/3216842549 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ec004960 con 0x559f6c5c0950 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f76f2a00640 1 -- 172.21.15.6:0/3216842549 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec005010 con 0x559f6c5c0950 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f76fe000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f76e80a2290 0x7f76e80a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f76fe000640 1 -- 172.21.15.6:0/1773248419 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f76f4014070 con 0x7f7700100370 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f7706600640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f76e80a2290 0x7f76e80a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f770b5a7740 1 -- 172.21.15.6:0/3216842549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 msgr2=0x559f6c5c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.160+0000 7f770b5a7740 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 0x559f6c5c2d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009900 tx=0x7f76ec02c6a0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 -- 172.21.15.6:0/3216842549 shutdown_connections 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5c0950 0x559f6c5c2d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 --2- 172.21.15.6:0/3216842549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b8140 0x559f6c5b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 -- 172.21.15.6:0/3216842549 >> 172.21.15.6:0/3216842549 conn(0x559f6c57a480 msgr2=0x559f6c5b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 -- 172.21.15.6:0/3216842549 shutdown_connections 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 -- 172.21.15.6:0/3216842549 wait complete. 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f7706600640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f76e80a2290 0x7f76e80a4750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009cf0 tx=0x7f76f8005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.161+0000 7f770b5a7740 1 -- start start 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 Processor -- start 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5bd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5bde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5be370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c5195a0 con 0x559f6c5b73d0 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c4beb20 con 0x559f6c5b8140 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c4be7c0 con 0x559f6c5c0950 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f770b5a7740 10 client.0 _ll_register_callbacks cb 0x559f6c4d2520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f7705c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5bd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f7705c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5bd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49874/0 (socket says 172.21.15.6:49874) 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f7705c00640 1 -- 172.21.15.6:0/1284521238 learned_addr learned my addr 172.21.15.6:0/1284521238 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f7705200640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5bde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.162+0000 7f7706600640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5be370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705c00640 1 -- 172.21.15.6:0/1284521238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 msgr2=0x559f6c5be370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705c00640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5be370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705c00640 1 -- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 msgr2=0x559f6c5bde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705c00640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5bde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705c00640 1 -- 172.21.15.6:0/1284521238 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770010d5b0 con 0x559f6c5b73d0 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705200640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f7705c00640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5bd8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f76f400cba0 tx=0x7f76f4009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6c07142adc55d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f76e7400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:30:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f76e7400640 1 -- 172.21.15.6:0/1284521238 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f409a780 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f76e7400640 1 -- 172.21.15.6:0/1284521238 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f76f4010dc0 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.163+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f6c4ebca0 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f76e7400640 1 -- 172.21.15.6:0/1284521238 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f409a8e0 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 10 client.15942 fetch_fsmap learned FSMap version 35 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559f6c517c20 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f76e7400640 1 -- 172.21.15.6:0/1284521238 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559f6c4ebca0 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f76e7400640 1 -- 172.21.15.6:0/1284521238 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f76f4099ff0 con 0x559f6c5b73d0 2024-02-26T06:30:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 10 client.15942 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 20 client.15942 populate_metadata read hostname 'smithi006' 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c5d0860 0x559f6c5d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 4 client.15942 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.164+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f6c50fcb0 con 0x559f6c5d0860 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.165+0000 7f7705200640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c5d0860 0x559f6c5d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.165+0000 7f7705200640 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c5d0860 0x559f6c5d2ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e100e0007d9c57 server_cookie=7358d9234ecdad53 in_seq=0 out_seq=0 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.165+0000 7f76e7400640 10 client.15942 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:30:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f76e7400640 1 -- 172.21.15.6:0/1284521238 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f6c50fcb0 con 0x559f6c5d0860 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f76e7400640 10 client.15942 handle_command_reply: tid=0 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 client.15942 shutdown 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 msgr2=0x559f6c5bd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5bd8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f76f400cba0 tx=0x7f76f4009200 comp rx=0 tx=0).stop 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 shutdown_connections 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c5d0860 0x559f6c5d2ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5be370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1284521238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b73d0 0x559f6c5bd8f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f76e7400640 0 client.15942 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 >> 172.21.15.6:0/1284521238 conn(0x559f6c57a480 msgr2=0x559f6c5b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 shutdown_connections 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.166+0000 7f770b5a7740 1 -- 172.21.15.6:0/1284521238 wait complete. 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.167+0000 7f770b5a7740 20 client.15942 trim_cache size 0 max 16384 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.181+0000 7f770b5a7740 1 Processor -- start 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.181+0000 7f770b5a7740 1 -- start start 2024-02-26T06:30:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b73d0 0x559f6c5c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 0x559f6c5bab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f770b5a7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c261e60 con 0x559f6c5c0950 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f770b5a7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c50fcb0 con 0x559f6c5b8140 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f770b5a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c5cd350 con 0x559f6c5b73d0 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7706600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 0x559f6c5bab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7706600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 0x559f6c5bab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49878/0 (socket says 172.21.15.6:49878) 2024-02-26T06:30:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7706600640 1 -- 172.21.15.6:0/1226799705 learned_addr learned my addr 172.21.15.6:0/1226799705 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7705200640 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7705c00640 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b73d0 0x559f6c5c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7706600640 1 -- 172.21.15.6:0/1226799705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b73d0 msgr2=0x559f6c5c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.182+0000 7f7706600640 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b73d0 0x559f6c5c0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f7706600640 1 -- 172.21.15.6:0/1226799705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 msgr2=0x559f6c5be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f7706600640 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f7706600640 1 -- 172.21.15.6:0/1226799705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f6c4b4500 con 0x559f6c5c0950 2024-02-26T06:30:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f7706600640 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 0x559f6c5bab90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f76f801f5e0 tx=0x7f76f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b3c840c572844c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f76f2000640 1 -- 172.21.15.6:0/1226799705 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8025040 con 0x559f6c5c0950 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f76f2000640 1 -- 172.21.15.6:0/1226799705 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f80119e0 con 0x559f6c5c0950 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.183+0000 7f76f2000640 1 -- 172.21.15.6:0/1226799705 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8017070 con 0x559f6c5c0950 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 -- 172.21.15.6:0/1226799705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 msgr2=0x559f6c5bab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 0x559f6c5bab90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f76f801f5e0 tx=0x7f76f801a000 comp rx=0 tx=0).stop 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 -- 172.21.15.6:0/1226799705 shutdown_connections 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5c0950 0x559f6c5bab90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b8140 0x559f6c5be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1226799705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5b73d0 0x559f6c5c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 -- 172.21.15.6:0/1226799705 >> 172.21.15.6:0/1226799705 conn(0x559f6c57a480 msgr2=0x559f6c5bc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 -- 172.21.15.6:0/1226799705 shutdown_connections 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.184+0000 7f770b5a7740 1 -- 172.21.15.6:0/1226799705 wait complete. 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 -- start start 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 Processor -- start 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b73d0 0x559f6c5c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 0x559f6c5c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c5c6c30 con 0x559f6c5b8140 2024-02-26T06:30:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c5c6da0 con 0x559f6c5b73d0 2024-02-26T06:30:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6c626d90 con 0x559f6c5c0950 2024-02-26T06:30:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.185+0000 7f770b5a7740 10 client.0 _ll_register_callbacks cb 0x7f76f400eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T06:30:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 0x559f6c5c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7706600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 0x559f6c5c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49888/0 (socket says 172.21.15.6:49888) 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 -- 172.21.15.6:0/1737837172 learned_addr learned my addr 172.21.15.6:0/1737837172 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705c00640 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b73d0 0x559f6c5c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 -- 172.21.15.6:0/1737837172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 msgr2=0x559f6c5c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 -- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b73d0 msgr2=0x559f6c5c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T06:30:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b73d0 0x559f6c5c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 -- 172.21.15.6:0/1737837172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f6c4da290 con 0x559f6c5b8140 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f7705200640 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 0x559f6c5c5f60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f76ec004530 tx=0x7f76ec0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f4067a9fdc95e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.186+0000 7f76e7400640 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f76e7400640 1 -- 172.21.15.6:0/1737837172 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec04e040 con 0x559f6c5b8140 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f76e7400640 1 -- 172.21.15.6:0/1737837172 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8549+0+0 (secure 0 0 0) 0x7f76ec050b00 con 0x559f6c5b8140 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f6c53ed00 con 0x559f6c5b8140 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f76e7400640 1 -- 172.21.15.6:0/1737837172 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76ec050cc0 con 0x559f6c5b8140 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f770b5a7740 10 client.15948 fetch_fsmap learned FSMap version 35 2024-02-26T06:30:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559f6c4c0b20 con 0x559f6c5b8140 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f76e7400640 1 -- 172.21.15.6:0/1737837172 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ec05e080 con 0x559f6c5b8140 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.187+0000 7f76e7400640 1 -- 172.21.15.6:0/1737837172 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 35) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f76ec03c050 con 0x559f6c5b8140 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f770b5a7740 10 client.15948 fetch_fsmap finished waiting for FSMap version 35 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f770b5a7740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f770b5a7740 20 client.15948 populate_metadata read hostname 'smithi006' 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c628360 0x559f6c62a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f770b5a7740 4 client.15948 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T06:30:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 --> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f6c219600 con 0x559f6c628360 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f7705c00640 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c628360 0x559f6c62a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f7705c00640 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c628360 0x559f6c62a780 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55779d96ea3930ea server_cookie=c987c0d638ee9f11 in_seq=0 out_seq=0 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.188+0000 7f76e7400640 10 client.15948 ms_handle_connect on v2:172.21.15.90:6838/137401855 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f76e7400640 1 -- 172.21.15.6:0/1737837172 <== mds.0 v2:172.21.15.90:6838/137401855 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f6c219600 con 0x559f6c628360 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f76e7400640 10 client.15948 handle_command_reply: tid=0 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f770b5a7740 1 client.15948 shutdown 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 msgr2=0x559f6c5c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 0x559f6c5c5f60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f76ec004530 tx=0x7f76ec0044c0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 shutdown_connections 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:6838/137401855,v1:172.21.15.90:6839/137401855] conn(0x559f6c628360 0x559f6c62a780 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.197+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f6c5c0950 0x559f6c5c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559f6c5b8140 0x559f6c5c5f60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f770b5a7740 1 --2- 172.21.15.6:0/1737837172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f6c5b73d0 0x559f6c5c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f76e7400640 0 client.15948 ms_handle_reset on v2:172.21.15.90:6838/137401855 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 >> 172.21.15.6:0/1737837172 conn(0x559f6c57a480 msgr2=0x559f6c5ac480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 shutdown_connections 2024-02-26T06:30:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f770b5a7740 1 -- 172.21.15.6:0/1737837172 wait complete. 2024-02-26T06:30:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.198+0000 7f770b5a7740 20 client.15948 trim_cache size 0 max 16384 2024-02-26T06:30:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2024-02-26T06:30:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2024-02-26T06:30:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2024-02-26T06:30:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2024-02-26T06:30:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f76e80a2290 msgr2=0x7f76e80a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f76e80a2290 0x7f76e80a4750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009cf0 tx=0x7f76f8005f90 comp rx=0 tx=0).stop 2024-02-26T06:30:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 msgr2=0x7f7700197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 0x7f7700197590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76f400ebc0 tx=0x7f76f400c6a0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 shutdown_connections 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f76e80a2290 0x7f76e80a4750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770019be50 0x7f770019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7700105240 0x7f7700197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 --2- 172.21.15.6:0/1773248419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7700100370 0x7f7700197590 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 >> 172.21.15.6:0/1773248419 conn(0x7f77000759e0 msgr2=0x7f7700103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 shutdown_connections 2024-02-26T06:30:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2024-02-26T06:30:08.199+0000 7f7707000640 1 -- 172.21.15.6:0/1773248419 wait complete. 2024-02-26T06:30:08.895 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-26T06:30:08.911 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-26T06:30:08.911 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T06:30:08.912 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:30:08.912 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-26T06:30:09.045 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-26T06:30:09.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:09 smithi167 ceph-mon[36229]: pgmap v64: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 197 KiB/s wr, 70 op/s 2024-02-26T06:30:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:09 smithi090 ceph-mon[35681]: pgmap v64: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 197 KiB/s wr, 70 op/s 2024-02-26T06:30:09.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:09 smithi006 ceph-mon[31790]: pgmap v64: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 197 KiB/s wr, 70 op/s 2024-02-26T06:30:11.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:11 smithi006 ceph-mon[31790]: pgmap v65: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 27 op/s 2024-02-26T06:30:11.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:11 smithi167 ceph-mon[36229]: pgmap v65: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 27 op/s 2024-02-26T06:30:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:11 smithi090 ceph-mon[35681]: pgmap v65: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 27 op/s 2024-02-26T06:30:13.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:13 smithi167 ceph-mon[36229]: pgmap v66: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 34 op/s 2024-02-26T06:30:13.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:13 smithi090 ceph-mon[35681]: pgmap v66: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 34 op/s 2024-02-26T06:30:13.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:13 smithi006 ceph-mon[31790]: pgmap v66: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 34 op/s 2024-02-26T06:30:15.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:15 smithi167 ceph-mon[36229]: pgmap v67: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-02-26T06:30:15.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:15 smithi090 ceph-mon[35681]: pgmap v67: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-02-26T06:30:15.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:15 smithi006 ceph-mon[31790]: pgmap v67: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-02-26T06:30:17.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:17 smithi167 ceph-mon[36229]: pgmap v68: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-02-26T06:30:17.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:17 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:30:17.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:17 smithi090 ceph-mon[35681]: pgmap v68: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-02-26T06:30:17.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:17 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:30:17.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:17 smithi006 ceph-mon[31790]: pgmap v68: 97 pgs: 97 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-02-26T06:30:17.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:17 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:30:18.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:18 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:30:18.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:18 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:30:18.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:18 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:30:18.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:18 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:30:18.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:18 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:30:18.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:18 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:30:18.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:18 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:30:18.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:18 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:30:18.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:18 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:30:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:19 smithi167 ceph-mon[36229]: pgmap v69: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 294 KiB/s wr, 37 op/s 2024-02-26T06:30:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:19 smithi090 ceph-mon[35681]: pgmap v69: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 294 KiB/s wr, 37 op/s 2024-02-26T06:30:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:19 smithi006 ceph-mon[31790]: pgmap v69: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 294 KiB/s wr, 37 op/s 2024-02-26T06:30:22.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:21 smithi167 ceph-mon[36229]: pgmap v70: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 284 KiB/s wr, 31 op/s 2024-02-26T06:30:22.523 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:21 smithi006 ceph-mon[31790]: pgmap v70: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 284 KiB/s wr, 31 op/s 2024-02-26T06:30:22.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:21 smithi090 ceph-mon[35681]: pgmap v70: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 284 KiB/s wr, 31 op/s 2024-02-26T06:30:24.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:23 smithi167 ceph-mon[36229]: pgmap v71: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 288 KiB/s wr, 35 op/s 2024-02-26T06:30:24.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:23 smithi090 ceph-mon[35681]: pgmap v71: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 288 KiB/s wr, 35 op/s 2024-02-26T06:30:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:23 smithi006 ceph-mon[31790]: pgmap v71: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 288 KiB/s wr, 35 op/s 2024-02-26T06:30:26.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:25 smithi167 ceph-mon[36229]: pgmap v72: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 283 KiB/s wr, 28 op/s 2024-02-26T06:30:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:25 smithi090 ceph-mon[35681]: pgmap v72: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 283 KiB/s wr, 28 op/s 2024-02-26T06:30:26.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:25 smithi006 ceph-mon[31790]: pgmap v72: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 283 KiB/s wr, 28 op/s 2024-02-26T06:30:28.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:27 smithi167 ceph-mon[36229]: pgmap v73: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 283 KiB/s wr, 28 op/s 2024-02-26T06:30:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:27 smithi090 ceph-mon[35681]: pgmap v73: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 283 KiB/s wr, 28 op/s 2024-02-26T06:30:28.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:27 smithi006 ceph-mon[31790]: pgmap v73: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 283 KiB/s wr, 28 op/s 2024-02-26T06:30:30.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:29 smithi167 ceph-mon[36229]: pgmap v74: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 286 KiB/s wr, 33 op/s 2024-02-26T06:30:30.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:29 smithi090 ceph-mon[35681]: pgmap v74: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 286 KiB/s wr, 33 op/s 2024-02-26T06:30:30.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:29 smithi006 ceph-mon[31790]: pgmap v74: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 286 KiB/s wr, 33 op/s 2024-02-26T06:30:32.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:31 smithi167 ceph-mon[36229]: pgmap v75: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-02-26T06:30:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:31 smithi090 ceph-mon[35681]: pgmap v75: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-02-26T06:30:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:31 smithi006 ceph-mon[31790]: pgmap v75: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-02-26T06:30:34.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:33 smithi167 ceph-mon[36229]: pgmap v76: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-02-26T06:30:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:33 smithi090 ceph-mon[35681]: pgmap v76: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-02-26T06:30:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:33 smithi006 ceph-mon[31790]: pgmap v76: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-02-26T06:30:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:35 smithi167 ceph-mon[36229]: pgmap v77: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-02-26T06:30:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:35 smithi090 ceph-mon[35681]: pgmap v77: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-02-26T06:30:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:35 smithi006 ceph-mon[31790]: pgmap v77: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-02-26T06:30:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:37 smithi167 ceph-mon[36229]: pgmap v78: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-02-26T06:30:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:37 smithi090 ceph-mon[35681]: pgmap v78: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-02-26T06:30:38.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:37 smithi006 ceph-mon[31790]: pgmap v78: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-02-26T06:30:40.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:39 smithi167 ceph-mon[36229]: pgmap v79: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 14 op/s 2024-02-26T06:30:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:39 smithi090 ceph-mon[35681]: pgmap v79: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 14 op/s 2024-02-26T06:30:40.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:39 smithi006 ceph-mon[31790]: pgmap v79: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 14 op/s 2024-02-26T06:30:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:41 smithi090 ceph-mon[35681]: pgmap v80: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-02-26T06:30:42.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:41 smithi006 ceph-mon[31790]: pgmap v80: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-02-26T06:30:42.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:41 smithi167 ceph-mon[36229]: pgmap v80: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-02-26T06:30:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:44 smithi090 ceph-mon[35681]: pgmap v81: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-02-26T06:30:44.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:44 smithi006 ceph-mon[31790]: pgmap v81: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-02-26T06:30:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:44 smithi167 ceph-mon[36229]: pgmap v81: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-02-26T06:30:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:46 smithi090 ceph-mon[35681]: pgmap v82: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-02-26T06:30:46.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:46 smithi006 ceph-mon[31790]: pgmap v82: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-02-26T06:30:46.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:46 smithi167 ceph-mon[36229]: pgmap v82: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-02-26T06:30:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:48 smithi090 ceph-mon[35681]: pgmap v83: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-02-26T06:30:48.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:48 smithi006 ceph-mon[31790]: pgmap v83: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-02-26T06:30:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:48 smithi167 ceph-mon[36229]: pgmap v83: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-02-26T06:30:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:50 smithi090 ceph-mon[35681]: pgmap v84: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-02-26T06:30:50.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:50 smithi006 ceph-mon[31790]: pgmap v84: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-02-26T06:30:50.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:50 smithi167 ceph-mon[36229]: pgmap v84: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-02-26T06:30:52.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:52 smithi090 ceph-mon[35681]: pgmap v85: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:52.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:52 smithi006 ceph-mon[31790]: pgmap v85: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:52.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:52 smithi167 ceph-mon[36229]: pgmap v85: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:54 smithi090 ceph-mon[35681]: pgmap v86: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-02-26T06:30:54.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:54 smithi006 ceph-mon[31790]: pgmap v86: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-02-26T06:30:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:54 smithi167 ceph-mon[36229]: pgmap v86: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-02-26T06:30:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:56 smithi090 ceph-mon[35681]: pgmap v87: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:56.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:56 smithi006 ceph-mon[31790]: pgmap v87: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:56.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:56 smithi167 ceph-mon[36229]: pgmap v87: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:30:58 smithi090 ceph-mon[35681]: pgmap v88: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:30:58 smithi006 ceph-mon[31790]: pgmap v88: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:30:58.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:30:58 smithi167 ceph-mon[36229]: pgmap v88: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T06:31:00.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:00 smithi167 ceph-mon[36229]: pgmap v89: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-02-26T06:31:00.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:00 smithi090 ceph-mon[35681]: pgmap v89: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-02-26T06:31:00.428 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:00 smithi006 ceph-mon[31790]: pgmap v89: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-02-26T06:31:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:02 smithi090 ceph-mon[35681]: pgmap v90: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:02 smithi006 ceph-mon[31790]: pgmap v90: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:02.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:02 smithi167 ceph-mon[36229]: pgmap v90: 97 pgs: 97 active+clean; 77 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:04 smithi090 ceph-mon[35681]: pgmap v91: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-02-26T06:31:04.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:04 smithi006 ceph-mon[31790]: pgmap v91: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-02-26T06:31:04.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:04 smithi167 ceph-mon[36229]: pgmap v91: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-02-26T06:31:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:06 smithi090 ceph-mon[35681]: pgmap v92: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-26T06:31:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:06 smithi006 ceph-mon[31790]: pgmap v92: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-26T06:31:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:06 smithi167 ceph-mon[36229]: pgmap v92: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-26T06:31:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:07 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:07.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:07 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:07.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:07 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:08 smithi090 ceph-mon[35681]: pgmap v93: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-26T06:31:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:08 smithi006 ceph-mon[31790]: pgmap v93: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-26T06:31:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:08 smithi167 ceph-mon[36229]: pgmap v93: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-26T06:31:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:10 smithi090 ceph-mon[35681]: pgmap v94: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-02-26T06:31:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:10 smithi006 ceph-mon[31790]: pgmap v94: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-02-26T06:31:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:10 smithi167 ceph-mon[36229]: pgmap v94: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-02-26T06:31:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:12 smithi090 ceph-mon[35681]: pgmap v95: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:31:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:12 smithi006 ceph-mon[31790]: pgmap v95: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:31:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:12 smithi167 ceph-mon[36229]: pgmap v95: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:31:13.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:13 smithi090 ceph-mon[35681]: pgmap v96: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-02-26T06:31:13.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:13 smithi006 ceph-mon[31790]: pgmap v96: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-02-26T06:31:13.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:13 smithi167 ceph-mon[36229]: pgmap v96: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-02-26T06:31:15.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:15 smithi090 ceph-mon[35681]: pgmap v97: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-02-26T06:31:15.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:15 smithi006 ceph-mon[31790]: pgmap v97: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-02-26T06:31:15.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:15 smithi167 ceph-mon[36229]: pgmap v97: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-02-26T06:31:17.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:17 smithi090 ceph-mon[35681]: pgmap v98: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-02-26T06:31:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:17 smithi006 ceph-mon[31790]: pgmap v98: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-02-26T06:31:17.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:17 smithi167 ceph-mon[36229]: pgmap v98: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-02-26T06:31:18.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:18 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:31:18.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:18 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:31:18.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:18 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:31:19.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:19 smithi090 ceph-mon[35681]: pgmap v99: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-02-26T06:31:19.674 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:19 smithi006 ceph-mon[31790]: pgmap v99: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-02-26T06:31:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:19 smithi167 ceph-mon[36229]: pgmap v99: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-02-26T06:31:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:20 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:31:20.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:20 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:31:20.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:20 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:31:20.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:20 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:31:20.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:20 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:31:20.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:20 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:31:20.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:20 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:31:20.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:20 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:31:20.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:20 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:31:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:21 smithi090 ceph-mon[35681]: pgmap v100: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-02-26T06:31:21.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:21 smithi006 ceph-mon[31790]: pgmap v100: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-02-26T06:31:21.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:21 smithi167 ceph-mon[36229]: pgmap v100: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-02-26T06:31:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:23 smithi090 ceph-mon[35681]: pgmap v101: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-02-26T06:31:23.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:23 smithi006 ceph-mon[31790]: pgmap v101: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-02-26T06:31:23.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:23 smithi167 ceph-mon[36229]: pgmap v101: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-02-26T06:31:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:25 smithi090 ceph-mon[35681]: pgmap v102: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:25.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:25 smithi006 ceph-mon[31790]: pgmap v102: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:25.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:25 smithi167 ceph-mon[36229]: pgmap v102: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:27.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:27 smithi090 ceph-mon[35681]: pgmap v103: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:27.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:27 smithi006 ceph-mon[31790]: pgmap v103: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:27.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:27 smithi167 ceph-mon[36229]: pgmap v103: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:29 smithi090 ceph-mon[35681]: pgmap v104: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-02-26T06:31:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:29 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:29.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:29 smithi006 ceph-mon[31790]: pgmap v104: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-02-26T06:31:29.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:29 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:29.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:29 smithi167 ceph-mon[36229]: pgmap v104: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-02-26T06:31:29.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:29 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:30.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:30 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:30.895 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:30 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:30.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:30 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:31.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:31 smithi090 ceph-mon[35681]: pgmap v105: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:31 smithi006 ceph-mon[31790]: pgmap v105: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:31.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:31 smithi167 ceph-mon[36229]: pgmap v105: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:33.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:33 smithi006 ceph-mon[31790]: pgmap v106: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-02-26T06:31:33.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:33 smithi167 ceph-mon[36229]: pgmap v106: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-02-26T06:31:34.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:33 smithi090 ceph-mon[35681]: pgmap v106: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-02-26T06:31:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:35 smithi006 ceph-mon[31790]: pgmap v107: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:35.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:35 smithi167 ceph-mon[36229]: pgmap v107: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:36.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:35 smithi090 ceph-mon[35681]: pgmap v107: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:37.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:37 smithi006 ceph-mon[31790]: pgmap v108: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:37.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:37 smithi167 ceph-mon[36229]: pgmap v108: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:38.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:37 smithi090 ceph-mon[35681]: pgmap v108: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-02-26T06:31:39.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:39 smithi006 ceph-mon[31790]: pgmap v109: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-02-26T06:31:39.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:39 smithi167 ceph-mon[36229]: pgmap v109: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-02-26T06:31:40.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:39 smithi090 ceph-mon[35681]: pgmap v109: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-02-26T06:31:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:41 smithi006 ceph-mon[31790]: pgmap v110: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-26T06:31:41.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:41 smithi167 ceph-mon[36229]: pgmap v110: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-26T06:31:42.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:41 smithi090 ceph-mon[35681]: pgmap v110: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-26T06:31:43.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:43 smithi006 ceph-mon[31790]: pgmap v111: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-02-26T06:31:43.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:43 smithi167 ceph-mon[36229]: pgmap v111: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-02-26T06:31:44.116 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:43 smithi090 ceph-mon[35681]: pgmap v111: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-02-26T06:31:45.917 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:45 smithi090 ceph-mon[35681]: pgmap v112: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:45.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:45 smithi006 ceph-mon[31790]: pgmap v112: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:45.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:45 smithi167 ceph-mon[36229]: pgmap v112: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:46.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:46 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:46.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:46 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:47.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:46 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:47.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:47 smithi167 ceph-mon[36229]: pgmap v113: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:48.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:47 smithi090 ceph-mon[35681]: pgmap v113: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:48.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:47 smithi006 ceph-mon[31790]: pgmap v113: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T06:31:49.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:49 smithi167 ceph-mon[36229]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:31:50.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:49 smithi090 ceph-mon[35681]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:31:50.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:49 smithi006 ceph-mon[31790]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-02-26T06:31:51.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:51 smithi167 ceph-mon[36229]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:31:52.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:51 smithi090 ceph-mon[35681]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:31:52.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:51 smithi006 ceph-mon[31790]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:31:53.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:53 smithi167 ceph-mon[36229]: pgmap v116: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:31:54.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:53 smithi090 ceph-mon[35681]: pgmap v116: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:31:54.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:53 smithi006 ceph-mon[31790]: pgmap v116: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:31:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:55 smithi167 ceph-mon[36229]: pgmap v117: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:31:56.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:55 smithi090 ceph-mon[35681]: pgmap v117: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:31:56.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:55 smithi006 ceph-mon[31790]: pgmap v117: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:31:57.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:57 smithi167 ceph-mon[36229]: pgmap v118: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:31:57.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:57 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:57 smithi090 ceph-mon[35681]: pgmap v118: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:31:58.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:57 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:57 smithi006 ceph-mon[31790]: pgmap v118: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:31:58.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:57 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:31:59.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:31:59 smithi167 ceph-mon[36229]: pgmap v119: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:00.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:31:59 smithi090 ceph-mon[35681]: pgmap v119: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:00.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:31:59 smithi006 ceph-mon[31790]: pgmap v119: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:01.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:01 smithi167 ceph-mon[36229]: pgmap v120: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:02.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:01 smithi090 ceph-mon[35681]: pgmap v120: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:02.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:01 smithi006 ceph-mon[31790]: pgmap v120: 97 pgs: 97 active+clean; 77 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:03.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:03 smithi167 ceph-mon[36229]: pgmap v121: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:04.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:03 smithi090 ceph-mon[35681]: pgmap v121: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:04.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:03 smithi006 ceph-mon[31790]: pgmap v121: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:06.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:05 smithi167 ceph-mon[36229]: pgmap v122: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:05 smithi090 ceph-mon[35681]: pgmap v122: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:05 smithi006 ceph-mon[31790]: pgmap v122: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:08.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:07 smithi167 ceph-mon[36229]: pgmap v123: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:07 smithi090 ceph-mon[35681]: pgmap v123: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:07 smithi006 ceph-mon[31790]: pgmap v123: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:10.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:09 smithi167 ceph-mon[36229]: pgmap v124: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:09 smithi090 ceph-mon[35681]: pgmap v124: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:09 smithi006 ceph-mon[31790]: pgmap v124: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:12.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:11 smithi167 ceph-mon[36229]: pgmap v125: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:11 smithi090 ceph-mon[35681]: pgmap v125: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:11 smithi006 ceph-mon[31790]: pgmap v125: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:32:14.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:13 smithi006 ceph-mon[31790]: pgmap v126: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:14.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:13 smithi167 ceph-mon[36229]: pgmap v126: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:14.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:13 smithi090 ceph-mon[35681]: pgmap v126: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:32:16.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:15 smithi167 ceph-mon[36229]: pgmap v127: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:32:16.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:15 smithi167 ceph-mon[36229]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:32:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:15 smithi090 ceph-mon[35681]: pgmap v127: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:32:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:15 smithi090 ceph-mon[35681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:32:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:15 smithi006 ceph-mon[31790]: pgmap v127: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:32:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:15 smithi006 ceph-mon[31790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-26T06:32:18.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:17 smithi167 ceph-mon[36229]: pgmap v128: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:32:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:17 smithi090 ceph-mon[35681]: pgmap v128: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:32:18.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:17 smithi006 ceph-mon[31790]: pgmap v128: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:32:20.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:19 smithi167 ceph-mon[36229]: pgmap v129: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:32:20.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:19 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:32:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:19 smithi090 ceph-mon[35681]: pgmap v129: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:32:20.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:19 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:32:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:19 smithi006 ceph-mon[31790]: pgmap v129: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:32:20.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:19 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:32:22.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:21 smithi167 ceph-mon[36229]: pgmap v130: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:22.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:21 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:32:22.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:21 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:32:22.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:21 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:32:22.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:21 smithi090 ceph-mon[35681]: pgmap v130: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:22.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:21 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:32:22.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:21 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:32:22.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:21 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:32:22.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:21 smithi006 ceph-mon[31790]: pgmap v130: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:22.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:21 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:32:22.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:21 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:32:22.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:21 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:32:24.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:23 smithi167 ceph-mon[36229]: pgmap v131: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:24.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:23 smithi090 ceph-mon[35681]: pgmap v131: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:23 smithi006 ceph-mon[31790]: pgmap v131: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:26.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:25 smithi167 ceph-mon[36229]: pgmap v132: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:25 smithi090 ceph-mon[35681]: pgmap v132: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:26.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:25 smithi006 ceph-mon[31790]: pgmap v132: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:27 smithi090 ceph-mon[35681]: pgmap v133: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:28.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:27 smithi006 ceph-mon[31790]: pgmap v133: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:28.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:27 smithi167 ceph-mon[36229]: pgmap v133: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:30.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:30 smithi090 ceph-mon[35681]: pgmap v134: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:30.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:30 smithi006 ceph-mon[31790]: pgmap v134: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:30.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:30 smithi167 ceph-mon[36229]: pgmap v134: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:32.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:32 smithi167 ceph-mon[36229]: pgmap v135: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:32 smithi090 ceph-mon[35681]: pgmap v135: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:32 smithi006 ceph-mon[31790]: pgmap v135: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:34 smithi090 ceph-mon[35681]: pgmap v136: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:34 smithi006 ceph-mon[31790]: pgmap v136: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:34.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:34 smithi167 ceph-mon[36229]: pgmap v136: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:36 smithi090 ceph-mon[35681]: pgmap v137: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:36 smithi006 ceph-mon[31790]: pgmap v137: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:36.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:36 smithi167 ceph-mon[36229]: pgmap v137: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:38.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:38 smithi090 ceph-mon[35681]: pgmap v138: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:38.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:38 smithi006 ceph-mon[31790]: pgmap v138: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:38.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:38 smithi167 ceph-mon[36229]: pgmap v138: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:40 smithi090 ceph-mon[35681]: pgmap v139: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:40.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:40 smithi006 ceph-mon[31790]: pgmap v139: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:40.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:40 smithi167 ceph-mon[36229]: pgmap v139: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:42 smithi090 ceph-mon[35681]: pgmap v140: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:42.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:42 smithi006 ceph-mon[31790]: pgmap v140: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:42.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:42 smithi167 ceph-mon[36229]: pgmap v140: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:44 smithi090 ceph-mon[35681]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:44.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:44 smithi006 ceph-mon[31790]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:44 smithi167 ceph-mon[36229]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:46 smithi090 ceph-mon[35681]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:46.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:46 smithi006 ceph-mon[31790]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:46.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:46 smithi167 ceph-mon[36229]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:48 smithi090 ceph-mon[35681]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:48.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:48 smithi006 ceph-mon[31790]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:48 smithi167 ceph-mon[36229]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:50 smithi090 ceph-mon[35681]: pgmap v144: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:50.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:50 smithi006 ceph-mon[31790]: pgmap v144: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:50.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:50 smithi167 ceph-mon[36229]: pgmap v144: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:52.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:52 smithi090 ceph-mon[35681]: pgmap v145: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:52.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:52 smithi006 ceph-mon[31790]: pgmap v145: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:52.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:52 smithi167 ceph-mon[36229]: pgmap v145: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:54 smithi090 ceph-mon[35681]: pgmap v146: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:54.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:54 smithi006 ceph-mon[31790]: pgmap v146: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:54 smithi167 ceph-mon[36229]: pgmap v146: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:32:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:56 smithi090 ceph-mon[35681]: pgmap v147: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:56.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:56 smithi006 ceph-mon[31790]: pgmap v147: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:56.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:56 smithi167 ceph-mon[36229]: pgmap v147: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:32:58 smithi090 ceph-mon[35681]: pgmap v148: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:32:58 smithi006 ceph-mon[31790]: pgmap v148: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:32:58.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:32:58 smithi167 ceph-mon[36229]: pgmap v148: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:00 smithi090 ceph-mon[35681]: pgmap v149: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:00 smithi006 ceph-mon[31790]: pgmap v149: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:00 smithi167 ceph-mon[36229]: pgmap v149: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:02 smithi090 ceph-mon[35681]: pgmap v150: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:02 smithi006 ceph-mon[31790]: pgmap v150: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:02.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:02 smithi167 ceph-mon[36229]: pgmap v150: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:04 smithi090 ceph-mon[35681]: pgmap v151: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:04.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:04 smithi006 ceph-mon[31790]: pgmap v151: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:04.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:04 smithi167 ceph-mon[36229]: pgmap v151: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:06 smithi090 ceph-mon[35681]: pgmap v152: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:06 smithi006 ceph-mon[31790]: pgmap v152: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:06 smithi167 ceph-mon[36229]: pgmap v152: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:08 smithi090 ceph-mon[35681]: pgmap v153: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:08 smithi006 ceph-mon[31790]: pgmap v153: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:08 smithi167 ceph-mon[36229]: pgmap v153: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:10 smithi090 ceph-mon[35681]: pgmap v154: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:10.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:10 smithi006 ceph-mon[31790]: pgmap v154: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:10 smithi167 ceph-mon[36229]: pgmap v154: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:12 smithi090 ceph-mon[35681]: pgmap v155: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:12 smithi006 ceph-mon[31790]: pgmap v155: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:12 smithi167 ceph-mon[36229]: pgmap v155: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:33:14.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:14 smithi006 ceph-mon[31790]: pgmap v156: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:14.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:14 smithi167 ceph-mon[36229]: pgmap v156: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:14.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:14 smithi090 ceph-mon[35681]: pgmap v156: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:33:15.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:15 smithi006 ceph-mon[31790]: pgmap v157: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:33:15.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:15 smithi167 ceph-mon[36229]: pgmap v157: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:33:15.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:15 smithi090 ceph-mon[35681]: pgmap v157: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:33:17.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:17 smithi006 ceph-mon[31790]: pgmap v158: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:33:17.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:17 smithi167 ceph-mon[36229]: pgmap v158: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:33:17.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:17 smithi090 ceph-mon[35681]: pgmap v158: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:33:19.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:19 smithi006 ceph-mon[31790]: pgmap v159: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:33:19.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:19 smithi167 ceph-mon[36229]: pgmap v159: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:33:19.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:19 smithi090 ceph-mon[35681]: pgmap v159: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:33:21.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:21 smithi006 ceph-mon[31790]: pgmap v160: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:21.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:21 smithi167 ceph-mon[36229]: pgmap v160: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:21 smithi090 ceph-mon[35681]: pgmap v160: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:22.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:22 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:33:22.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:22 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:33:22.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:22 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:33:23.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:23 smithi006 ceph-mon[31790]: pgmap v161: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:23 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:33:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:23 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:33:23.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:23 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:33:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:23 smithi167 ceph-mon[36229]: pgmap v161: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:23 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:33:23.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:23 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:33:23.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:23 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:33:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:23 smithi090 ceph-mon[35681]: pgmap v161: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:23 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:33:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:23 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:33:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:23 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:33:25.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:25 smithi006 ceph-mon[31790]: pgmap v162: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:25.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:25 smithi167 ceph-mon[36229]: pgmap v162: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:25 smithi090 ceph-mon[35681]: pgmap v162: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:27.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:27 smithi167 ceph-mon[36229]: pgmap v163: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:27.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:27 smithi090 ceph-mon[35681]: pgmap v163: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:27.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:27 smithi006 ceph-mon[31790]: pgmap v163: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:29.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:29 smithi167 ceph-mon[36229]: pgmap v164: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:29 smithi090 ceph-mon[35681]: pgmap v164: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:29.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:29 smithi006 ceph-mon[31790]: pgmap v164: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:31 smithi167 ceph-mon[36229]: pgmap v165: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:31.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:31 smithi090 ceph-mon[35681]: pgmap v165: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:31.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:31 smithi006 ceph-mon[31790]: pgmap v165: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:33 smithi167 ceph-mon[36229]: pgmap v166: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:33 smithi090 ceph-mon[35681]: pgmap v166: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:33.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:33 smithi006 ceph-mon[31790]: pgmap v166: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:35.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:35 smithi167 ceph-mon[36229]: pgmap v167: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:35.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:35 smithi090 ceph-mon[35681]: pgmap v167: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:35.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:35 smithi006 ceph-mon[31790]: pgmap v167: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:37 smithi167 ceph-mon[36229]: pgmap v168: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:37.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:37 smithi090 ceph-mon[35681]: pgmap v168: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:37.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:37 smithi006 ceph-mon[31790]: pgmap v168: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:39.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:39 smithi167 ceph-mon[36229]: pgmap v169: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:39.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:39 smithi090 ceph-mon[35681]: pgmap v169: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:39.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:39 smithi006 ceph-mon[31790]: pgmap v169: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:41.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:41 smithi167 ceph-mon[36229]: pgmap v170: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:41.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:41 smithi090 ceph-mon[35681]: pgmap v170: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:41 smithi006 ceph-mon[31790]: pgmap v170: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:43.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:43 smithi167 ceph-mon[36229]: pgmap v171: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:43.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:43 smithi090 ceph-mon[35681]: pgmap v171: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:43.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:43 smithi006 ceph-mon[31790]: pgmap v171: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:45.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:45 smithi006 ceph-mon[31790]: pgmap v172: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:45.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:45 smithi167 ceph-mon[36229]: pgmap v172: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:45.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:45 smithi090 ceph-mon[35681]: pgmap v172: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:47.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:47 smithi167 ceph-mon[36229]: pgmap v173: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:47 smithi090 ceph-mon[35681]: pgmap v173: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:47.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:47 smithi006 ceph-mon[31790]: pgmap v173: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:49.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:49 smithi167 ceph-mon[36229]: pgmap v174: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:49.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:49 smithi090 ceph-mon[35681]: pgmap v174: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:49.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:49 smithi006 ceph-mon[31790]: pgmap v174: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:51.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:51 smithi167 ceph-mon[36229]: pgmap v175: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:51 smithi090 ceph-mon[35681]: pgmap v175: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:51 smithi006 ceph-mon[31790]: pgmap v175: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:53.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:53 smithi167 ceph-mon[36229]: pgmap v176: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:53.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:53 smithi090 ceph-mon[35681]: pgmap v176: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:53.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:53 smithi006 ceph-mon[31790]: pgmap v176: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:55.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:55 smithi167 ceph-mon[36229]: pgmap v177: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:55.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:55 smithi090 ceph-mon[35681]: pgmap v177: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:55.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:55 smithi006 ceph-mon[31790]: pgmap v177: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:57 smithi090 ceph-mon[35681]: pgmap v178: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:57.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:57 smithi006 ceph-mon[31790]: pgmap v178: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:57.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:57 smithi167 ceph-mon[36229]: pgmap v178: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:33:59.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:33:59 smithi090 ceph-mon[35681]: pgmap v179: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:59.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:33:59 smithi006 ceph-mon[31790]: pgmap v179: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:33:59.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:33:59 smithi167 ceph-mon[36229]: pgmap v179: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:01.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:01 smithi090 ceph-mon[35681]: pgmap v180: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:01.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:01 smithi006 ceph-mon[31790]: pgmap v180: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:01.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:01 smithi167 ceph-mon[36229]: pgmap v180: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:03.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:03 smithi167 ceph-mon[36229]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:03.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:03 smithi090 ceph-mon[35681]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:03.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:03 smithi006 ceph-mon[31790]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:05.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:05 smithi090 ceph-mon[35681]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:05.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:05 smithi006 ceph-mon[31790]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:05.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:05 smithi167 ceph-mon[36229]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:07.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:07 smithi090 ceph-mon[35681]: pgmap v183: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:07.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:07 smithi006 ceph-mon[31790]: pgmap v183: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:07.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:07 smithi167 ceph-mon[36229]: pgmap v183: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:09 smithi090 ceph-mon[35681]: pgmap v184: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:09.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:09 smithi006 ceph-mon[31790]: pgmap v184: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:09.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:09 smithi167 ceph-mon[36229]: pgmap v184: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:11.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:11 smithi090 ceph-mon[35681]: pgmap v185: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:11.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:11 smithi006 ceph-mon[31790]: pgmap v185: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:11.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:11 smithi167 ceph-mon[36229]: pgmap v185: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-02-26T06:34:13.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:13 smithi090 ceph-mon[35681]: pgmap v186: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:13.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:13 smithi006 ceph-mon[31790]: pgmap v186: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:13.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:13 smithi167 ceph-mon[36229]: pgmap v186: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-02-26T06:34:15.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:15 smithi090 ceph-mon[35681]: pgmap v187: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:34:15.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:15 smithi006 ceph-mon[31790]: pgmap v187: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:34:15.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:15 smithi167 ceph-mon[36229]: pgmap v187: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:34:17.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:17 smithi090 ceph-mon[35681]: pgmap v188: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:34:17.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:17 smithi006 ceph-mon[31790]: pgmap v188: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:34:17.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:17 smithi167 ceph-mon[36229]: pgmap v188: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-26T06:34:19.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:19 smithi090 ceph-mon[35681]: pgmap v189: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:34:19.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:19 smithi006 ceph-mon[31790]: pgmap v189: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:34:19.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:19 smithi167 ceph-mon[36229]: pgmap v189: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:34:21.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:21 smithi090 ceph-mon[35681]: pgmap v190: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:21.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:21 smithi006 ceph-mon[31790]: pgmap v190: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:21.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:21 smithi167 ceph-mon[36229]: pgmap v190: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:23 smithi090 ceph-mon[35681]: pgmap v191: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:23.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:23 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:34:23.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:23 smithi006 ceph-mon[31790]: pgmap v191: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:23.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:23 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:34:23.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:23 smithi167 ceph-mon[36229]: pgmap v191: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:23.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:23 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:34:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:25 smithi090 ceph-mon[35681]: pgmap v192: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:25 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:34:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:25 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:34:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:25 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:34:25.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:25 smithi006 ceph-mon[31790]: pgmap v192: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:25.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:25 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:34:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:25 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:34:25.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:25 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:34:25.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:25 smithi167 ceph-mon[36229]: pgmap v192: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:25.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:25 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:34:25.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:25 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:34:25.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:25 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:34:27.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:27 smithi090 ceph-mon[35681]: pgmap v193: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:27.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:27 smithi006 ceph-mon[31790]: pgmap v193: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:27.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:27 smithi167 ceph-mon[36229]: pgmap v193: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:29.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:29 smithi090 ceph-mon[35681]: pgmap v194: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:29.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:29 smithi006 ceph-mon[31790]: pgmap v194: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:29.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:29 smithi167 ceph-mon[36229]: pgmap v194: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:32.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:31 smithi167 ceph-mon[36229]: pgmap v195: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:31 smithi090 ceph-mon[35681]: pgmap v195: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:31 smithi006 ceph-mon[31790]: pgmap v195: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:34.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:33 smithi167 ceph-mon[36229]: pgmap v196: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:33 smithi090 ceph-mon[35681]: pgmap v196: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:33 smithi006 ceph-mon[31790]: pgmap v196: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:36 smithi090 ceph-mon[35681]: pgmap v197: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:35 smithi006 ceph-mon[31790]: pgmap v197: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:36.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:36 smithi167 ceph-mon[36229]: pgmap v197: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:38.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:38 smithi090 ceph-mon[35681]: pgmap v198: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:38.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:38 smithi006 ceph-mon[31790]: pgmap v198: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:38 smithi167 ceph-mon[36229]: pgmap v198: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:40 smithi090 ceph-mon[35681]: pgmap v199: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:40.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:40 smithi006 ceph-mon[31790]: pgmap v199: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:40.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:40 smithi167 ceph-mon[36229]: pgmap v199: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:42 smithi090 ceph-mon[35681]: pgmap v200: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:42.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:42 smithi006 ceph-mon[31790]: pgmap v200: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:42.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:42 smithi167 ceph-mon[36229]: pgmap v200: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:44.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:44 smithi090 ceph-mon[35681]: pgmap v201: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:44.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:44 smithi006 ceph-mon[31790]: pgmap v201: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:44 smithi167 ceph-mon[36229]: pgmap v201: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:46 smithi090 ceph-mon[35681]: pgmap v202: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:46.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:46 smithi006 ceph-mon[31790]: pgmap v202: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:46.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:46 smithi167 ceph-mon[36229]: pgmap v202: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:48 smithi090 ceph-mon[35681]: pgmap v203: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:48.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:48 smithi006 ceph-mon[31790]: pgmap v203: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:48 smithi167 ceph-mon[36229]: pgmap v203: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:50.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:50 smithi090 ceph-mon[35681]: pgmap v204: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:50.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:50 smithi006 ceph-mon[31790]: pgmap v204: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:50.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:50 smithi167 ceph-mon[36229]: pgmap v204: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:52.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:52 smithi090 ceph-mon[35681]: pgmap v205: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:52.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:52 smithi006 ceph-mon[31790]: pgmap v205: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:52.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:52 smithi167 ceph-mon[36229]: pgmap v205: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:54 smithi090 ceph-mon[35681]: pgmap v206: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:54.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:54 smithi006 ceph-mon[31790]: pgmap v206: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:54 smithi167 ceph-mon[36229]: pgmap v206: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:34:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:56 smithi090 ceph-mon[35681]: pgmap v207: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:56.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:56 smithi006 ceph-mon[31790]: pgmap v207: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:56.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:56 smithi167 ceph-mon[36229]: pgmap v207: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:58.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:34:58 smithi090 ceph-mon[35681]: pgmap v208: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:34:58 smithi006 ceph-mon[31790]: pgmap v208: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:34:58.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:34:58 smithi167 ceph-mon[36229]: pgmap v208: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:00 smithi090 ceph-mon[35681]: pgmap v209: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:00 smithi006 ceph-mon[31790]: pgmap v209: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:00 smithi167 ceph-mon[36229]: pgmap v209: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:02.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:02 smithi090 ceph-mon[35681]: pgmap v210: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:02 smithi006 ceph-mon[31790]: pgmap v210: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:02.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:02 smithi167 ceph-mon[36229]: pgmap v210: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:04.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:04 smithi090 ceph-mon[35681]: pgmap v211: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:35:04.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:04 smithi006 ceph-mon[31790]: pgmap v211: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:35:04.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:04 smithi167 ceph-mon[36229]: pgmap v211: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:35:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:06 smithi090 ceph-mon[35681]: pgmap v212: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:35:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:06 smithi006 ceph-mon[31790]: pgmap v212: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:35:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:06 smithi167 ceph-mon[36229]: pgmap v212: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:35:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:07 smithi090 ceph-mon[35681]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T06:35:07.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:07 smithi006 ceph-mon[31790]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T06:35:07.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:07 smithi167 ceph-mon[36229]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T06:35:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:08 smithi090 ceph-mon[35681]: pgmap v214: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:35:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:08 smithi006 ceph-mon[31790]: pgmap v214: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:35:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:08 smithi167 ceph-mon[36229]: pgmap v214: 97 pgs: 97 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:35:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:10 smithi090 ceph-mon[35681]: pgmap v215: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:35:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:10 smithi090 ceph-mon[35681]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T06:35:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:10 smithi006 ceph-mon[31790]: pgmap v215: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:35:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:10 smithi006 ceph-mon[31790]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T06:35:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:10 smithi167 ceph-mon[36229]: pgmap v215: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:35:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:10 smithi167 ceph-mon[36229]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T06:35:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:12 smithi006 ceph-mon[31790]: pgmap v217: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:35:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:12 smithi167 ceph-mon[36229]: pgmap v217: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:35:12.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:12 smithi090 ceph-mon[35681]: pgmap v217: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:35:14.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:14 smithi006 ceph-mon[31790]: pgmap v218: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:35:14.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:14 smithi167 ceph-mon[36229]: pgmap v218: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:35:14.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:14 smithi090 ceph-mon[35681]: pgmap v218: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:35:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:16 smithi006 ceph-mon[31790]: pgmap v219: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-02-26T06:35:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:16 smithi167 ceph-mon[36229]: pgmap v219: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-02-26T06:35:16.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:16 smithi090 ceph-mon[35681]: pgmap v219: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-02-26T06:35:18.417 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:18 smithi090 ceph-mon[35681]: pgmap v220: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:35:18.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:18 smithi006 ceph-mon[31790]: pgmap v220: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:35:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:18 smithi167 ceph-mon[36229]: pgmap v220: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:35:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:20 smithi006 ceph-mon[31790]: pgmap v221: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:35:20.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:20 smithi167 ceph-mon[36229]: pgmap v221: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:35:20.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:20 smithi090 ceph-mon[35681]: pgmap v221: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:35:22.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:22 smithi006 ceph-mon[31790]: pgmap v222: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-02-26T06:35:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:22 smithi167 ceph-mon[36229]: pgmap v222: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-02-26T06:35:22.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:22 smithi090 ceph-mon[35681]: pgmap v222: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-02-26T06:35:23.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:23 smithi006 ceph-mon[31790]: pgmap v223: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:35:23.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:23 smithi167 ceph-mon[36229]: pgmap v223: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:35:23.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:23 smithi090 ceph-mon[35681]: pgmap v223: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:35:25.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:25 smithi167 ceph-mon[36229]: pgmap v224: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:25.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:25 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:35:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:25 smithi090 ceph-mon[35681]: pgmap v224: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:25.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:25 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:35:25.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:25 smithi006 ceph-mon[31790]: pgmap v224: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:25.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:25 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:35:26.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:26 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:35:26.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:26 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:35:26.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:26 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:35:26.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:26 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:35:26.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:26 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:35:26.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:26 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:35:26.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:26 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:35:26.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:26 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:35:26.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:26 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:35:27.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:27 smithi167 ceph-mon[36229]: pgmap v225: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:27.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:27 smithi090 ceph-mon[35681]: pgmap v225: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:27.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:27 smithi006 ceph-mon[31790]: pgmap v225: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:29.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:29 smithi167 ceph-mon[36229]: pgmap v226: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:29.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:29 smithi090 ceph-mon[35681]: pgmap v226: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:29.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:29 smithi006 ceph-mon[31790]: pgmap v226: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:31.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:31 smithi167 ceph-mon[36229]: pgmap v227: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:31.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:31 smithi090 ceph-mon[35681]: pgmap v227: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:31.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:31 smithi006 ceph-mon[31790]: pgmap v227: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:33 smithi167 ceph-mon[36229]: pgmap v228: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:33.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:33 smithi090 ceph-mon[35681]: pgmap v228: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:33.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:33 smithi006 ceph-mon[31790]: pgmap v228: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:35.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:35 smithi167 ceph-mon[36229]: pgmap v229: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:35.871 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:35 smithi090 ceph-mon[35681]: pgmap v229: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:35.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:35 smithi006 ceph-mon[31790]: pgmap v229: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:37.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:37 smithi167 ceph-mon[36229]: pgmap v230: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:37.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:37 smithi090 ceph-mon[35681]: pgmap v230: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:37.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:37 smithi006 ceph-mon[31790]: pgmap v230: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:39.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:39 smithi167 ceph-mon[36229]: pgmap v231: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:39.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:39 smithi090 ceph-mon[35681]: pgmap v231: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:39.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:39 smithi006 ceph-mon[31790]: pgmap v231: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:41.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:41 smithi167 ceph-mon[36229]: pgmap v232: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:41.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:41 smithi090 ceph-mon[35681]: pgmap v232: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:41.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:41 smithi006 ceph-mon[31790]: pgmap v232: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:43.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:43 smithi090 ceph-mon[35681]: pgmap v233: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:43.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:43 smithi006 ceph-mon[31790]: pgmap v233: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:43.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:43 smithi167 ceph-mon[36229]: pgmap v233: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:45.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:45 smithi090 ceph-mon[35681]: pgmap v234: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:45.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:45 smithi006 ceph-mon[31790]: pgmap v234: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:45.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:45 smithi167 ceph-mon[36229]: pgmap v234: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:47.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:47 smithi090 ceph-mon[35681]: pgmap v235: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:47.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:47 smithi006 ceph-mon[31790]: pgmap v235: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:47.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:47 smithi167 ceph-mon[36229]: pgmap v235: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:49.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:49 smithi090 ceph-mon[35681]: pgmap v236: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:49.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:49 smithi006 ceph-mon[31790]: pgmap v236: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:49.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:49 smithi167 ceph-mon[36229]: pgmap v236: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:51.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:51 smithi090 ceph-mon[35681]: pgmap v237: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:51 smithi006 ceph-mon[31790]: pgmap v237: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:51.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:51 smithi167 ceph-mon[36229]: pgmap v237: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:35:53.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:53 smithi090 ceph-mon[35681]: pgmap v238: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:53.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:53 smithi006 ceph-mon[31790]: pgmap v238: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:53.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:53 smithi167 ceph-mon[36229]: pgmap v238: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:55.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:55 smithi090 ceph-mon[35681]: pgmap v239: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:35:55.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:55 smithi006 ceph-mon[31790]: pgmap v239: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:35:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:55 smithi167 ceph-mon[36229]: pgmap v239: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:35:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:57 smithi090 ceph-mon[35681]: pgmap v240: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:35:57.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:57 smithi006 ceph-mon[31790]: pgmap v240: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:35:57.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:57 smithi167 ceph-mon[36229]: pgmap v240: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:35:59.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:35:59 smithi090 ceph-mon[35681]: pgmap v241: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:59.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:35:59 smithi006 ceph-mon[31790]: pgmap v241: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:35:59.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:35:59 smithi167 ceph-mon[36229]: pgmap v241: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:01.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:01 smithi090 ceph-mon[35681]: pgmap v242: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:01.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:01 smithi006 ceph-mon[31790]: pgmap v242: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:01.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:01 smithi167 ceph-mon[36229]: pgmap v242: 97 pgs: 97 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:03.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:03 smithi090 ceph-mon[35681]: pgmap v243: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:36:03.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:03 smithi006 ceph-mon[31790]: pgmap v243: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:36:03.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:03 smithi167 ceph-mon[36229]: pgmap v243: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:36:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:06 smithi090 ceph-mon[35681]: pgmap v244: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:36:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:06 smithi006 ceph-mon[31790]: pgmap v244: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:36:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:06 smithi167 ceph-mon[36229]: pgmap v244: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:36:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:08 smithi090 ceph-mon[35681]: pgmap v245: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:36:08.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:08 smithi090 ceph-mon[35681]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T06:36:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:08 smithi006 ceph-mon[31790]: pgmap v245: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:36:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:08 smithi006 ceph-mon[31790]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T06:36:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:08 smithi167 ceph-mon[36229]: pgmap v245: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:36:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:08 smithi167 ceph-mon[36229]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T06:36:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:10 smithi090 ceph-mon[35681]: pgmap v247: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:36:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:10 smithi006 ceph-mon[31790]: pgmap v247: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:36:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:10 smithi167 ceph-mon[36229]: pgmap v247: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:36:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:12 smithi090 ceph-mon[35681]: pgmap v248: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:36:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:12 smithi006 ceph-mon[31790]: pgmap v248: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:36:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:12 smithi167 ceph-mon[36229]: pgmap v248: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:36:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:14 smithi090 ceph-mon[35681]: pgmap v249: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:36:14.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:14 smithi006 ceph-mon[31790]: pgmap v249: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:36:14.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:14 smithi167 ceph-mon[36229]: pgmap v249: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:36:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:16 smithi090 ceph-mon[35681]: pgmap v250: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:36:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:16 smithi090 ceph-mon[35681]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T06:36:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:16 smithi006 ceph-mon[31790]: pgmap v250: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:36:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:16 smithi006 ceph-mon[31790]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T06:36:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:16 smithi167 ceph-mon[36229]: pgmap v250: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:36:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:16 smithi167 ceph-mon[36229]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T06:36:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:18 smithi090 ceph-mon[35681]: pgmap v252: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 218 B/s wr, 11 op/s 2024-02-26T06:36:18.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:18 smithi006 ceph-mon[31790]: pgmap v252: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 218 B/s wr, 11 op/s 2024-02-26T06:36:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:18 smithi167 ceph-mon[36229]: pgmap v252: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 218 B/s wr, 11 op/s 2024-02-26T06:36:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:20 smithi090 ceph-mon[35681]: pgmap v253: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:36:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:20 smithi006 ceph-mon[31790]: pgmap v253: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:36:20.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:20 smithi167 ceph-mon[36229]: pgmap v253: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:36:22.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:22 smithi090 ceph-mon[35681]: pgmap v254: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:36:22.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:22 smithi006 ceph-mon[31790]: pgmap v254: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:36:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:22 smithi167 ceph-mon[36229]: pgmap v254: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:36:24.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:24 smithi090 ceph-mon[35681]: pgmap v255: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-26T06:36:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:24 smithi006 ceph-mon[31790]: pgmap v255: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-26T06:36:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:24 smithi167 ceph-mon[36229]: pgmap v255: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-26T06:36:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:26 smithi090 ceph-mon[35681]: pgmap v256: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-26T06:36:26.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:26 smithi006 ceph-mon[31790]: pgmap v256: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-26T06:36:26.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:26 smithi167 ceph-mon[36229]: pgmap v256: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-26T06:36:27.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:27 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:36:27.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:27 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:36:27.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:27 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:36:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:28 smithi090 ceph-mon[35681]: pgmap v257: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-02-26T06:36:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:28 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:36:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:28 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:36:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:28 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:36:28.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:28 smithi006 ceph-mon[31790]: pgmap v257: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-02-26T06:36:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:28 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:36:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:28 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:36:28.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:28 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:36:28.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:28 smithi167 ceph-mon[36229]: pgmap v257: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-02-26T06:36:28.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:28 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:36:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:28 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:36:28.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:28 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:36:30.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:30 smithi090 ceph-mon[35681]: pgmap v258: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:30.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:30 smithi006 ceph-mon[31790]: pgmap v258: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:30.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:30 smithi167 ceph-mon[36229]: pgmap v258: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:32.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:32 smithi090 ceph-mon[35681]: pgmap v259: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:32 smithi006 ceph-mon[31790]: pgmap v259: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:32.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:32 smithi167 ceph-mon[36229]: pgmap v259: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:34.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:34 smithi090 ceph-mon[35681]: pgmap v260: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:34 smithi006 ceph-mon[31790]: pgmap v260: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:34.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:34 smithi167 ceph-mon[36229]: pgmap v260: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:36.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:36 smithi090 ceph-mon[35681]: pgmap v261: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:36.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:36 smithi006 ceph-mon[31790]: pgmap v261: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:36.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:36 smithi167 ceph-mon[36229]: pgmap v261: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:38.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:38 smithi090 ceph-mon[35681]: pgmap v262: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:38.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:38 smithi006 ceph-mon[31790]: pgmap v262: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:38 smithi167 ceph-mon[36229]: pgmap v262: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:40.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:40 smithi090 ceph-mon[35681]: pgmap v263: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:40.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:40 smithi006 ceph-mon[31790]: pgmap v263: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:40.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:40 smithi167 ceph-mon[36229]: pgmap v263: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:42 smithi090 ceph-mon[35681]: pgmap v264: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:42.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:42 smithi006 ceph-mon[31790]: pgmap v264: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:42.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:42 smithi167 ceph-mon[36229]: pgmap v264: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:44.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:44 smithi090 ceph-mon[35681]: pgmap v265: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:44.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:44 smithi006 ceph-mon[31790]: pgmap v265: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:44 smithi167 ceph-mon[36229]: pgmap v265: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:46.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:46 smithi090 ceph-mon[35681]: pgmap v266: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:46.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:46 smithi006 ceph-mon[31790]: pgmap v266: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:46.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:46 smithi167 ceph-mon[36229]: pgmap v266: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:48.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:48 smithi006 ceph-mon[31790]: pgmap v267: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:48 smithi167 ceph-mon[36229]: pgmap v267: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:48.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:48 smithi090 ceph-mon[35681]: pgmap v267: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:50.417 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:50 smithi090 ceph-mon[35681]: pgmap v268: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:50.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:50 smithi006 ceph-mon[31790]: pgmap v268: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:50.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:50 smithi167 ceph-mon[36229]: pgmap v268: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:52.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:52 smithi006 ceph-mon[31790]: pgmap v269: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:52.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:52 smithi167 ceph-mon[36229]: pgmap v269: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:52.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:52 smithi090 ceph-mon[35681]: pgmap v269: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:36:54.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:54 smithi006 ceph-mon[31790]: pgmap v270: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:54 smithi167 ceph-mon[36229]: pgmap v270: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:54.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:54 smithi090 ceph-mon[35681]: pgmap v270: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:36:56.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:56 smithi006 ceph-mon[31790]: pgmap v271: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:56.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:56 smithi167 ceph-mon[36229]: pgmap v271: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:56.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:56 smithi090 ceph-mon[35681]: pgmap v271: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:58.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:36:58 smithi006 ceph-mon[31790]: pgmap v272: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:58.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:36:58 smithi167 ceph-mon[36229]: pgmap v272: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:36:58.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:36:58 smithi090 ceph-mon[35681]: pgmap v272: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:37:00.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:00 smithi006 ceph-mon[31790]: pgmap v273: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:00 smithi167 ceph-mon[36229]: pgmap v273: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:00.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:00 smithi090 ceph-mon[35681]: pgmap v273: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:02.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:02 smithi006 ceph-mon[31790]: pgmap v274: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:37:02.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:02 smithi167 ceph-mon[36229]: pgmap v274: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:37:02.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:02 smithi090 ceph-mon[35681]: pgmap v274: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-26T06:37:04.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:04 smithi006 ceph-mon[31790]: pgmap v275: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:37:04.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:04 smithi167 ceph-mon[36229]: pgmap v275: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:37:04.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:04 smithi090 ceph-mon[35681]: pgmap v275: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:37:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:06 smithi006 ceph-mon[31790]: pgmap v276: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:37:06.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:06 smithi167 ceph-mon[36229]: pgmap v276: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:37:06.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:06 smithi090 ceph-mon[35681]: pgmap v276: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:37:07.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:07 smithi167 ceph-mon[36229]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T06:37:07.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:07 smithi167 ceph-mon[36229]: pgmap v278: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:37:07.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:07 smithi090 ceph-mon[35681]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T06:37:07.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:07 smithi090 ceph-mon[35681]: pgmap v278: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:37:07.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:07 smithi006 ceph-mon[31790]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T06:37:07.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:07 smithi006 ceph-mon[31790]: pgmap v278: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:37:09.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:09 smithi090 ceph-mon[35681]: pgmap v279: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:37:09.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:09 smithi006 ceph-mon[31790]: pgmap v279: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:37:09.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:09 smithi167 ceph-mon[36229]: pgmap v279: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:37:11.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:11 smithi090 ceph-mon[35681]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T06:37:11.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:11 smithi006 ceph-mon[31790]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T06:37:11.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:11 smithi167 ceph-mon[36229]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T06:37:12.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:12 smithi090 ceph-mon[35681]: pgmap v281: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:37:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:12 smithi006 ceph-mon[31790]: pgmap v281: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:37:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:12 smithi167 ceph-mon[36229]: pgmap v281: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:37:14.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:14 smithi090 ceph-mon[35681]: pgmap v282: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:37:14.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:14 smithi006 ceph-mon[31790]: pgmap v282: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:37:14.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:14 smithi167 ceph-mon[36229]: pgmap v282: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-26T06:37:16.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:16 smithi090 ceph-mon[35681]: pgmap v283: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-02-26T06:37:16.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:16 smithi006 ceph-mon[31790]: pgmap v283: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-02-26T06:37:16.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:16 smithi167 ceph-mon[36229]: pgmap v283: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 247 B/s wr, 12 op/s 2024-02-26T06:37:18.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:18 smithi090 ceph-mon[35681]: pgmap v284: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:37:18.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:18 smithi006 ceph-mon[31790]: pgmap v284: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:37:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:18 smithi167 ceph-mon[36229]: pgmap v284: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-26T06:37:20.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:20 smithi090 ceph-mon[35681]: pgmap v285: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:37:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:20 smithi006 ceph-mon[31790]: pgmap v285: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:37:20.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:20 smithi167 ceph-mon[36229]: pgmap v285: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-02-26T06:37:22.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:22 smithi090 ceph-mon[35681]: pgmap v286: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-02-26T06:37:22.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:22 smithi006 ceph-mon[31790]: pgmap v286: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-02-26T06:37:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:22 smithi167 ceph-mon[36229]: pgmap v286: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-02-26T06:37:24.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:24 smithi090 ceph-mon[35681]: pgmap v287: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:37:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:24 smithi006 ceph-mon[31790]: pgmap v287: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:37:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:24 smithi167 ceph-mon[36229]: pgmap v287: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-26T06:37:26.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:26 smithi090 ceph-mon[35681]: pgmap v288: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:26.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:26 smithi006 ceph-mon[31790]: pgmap v288: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:26.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:26 smithi167 ceph-mon[36229]: pgmap v288: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:28.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:28 smithi090 ceph-mon[35681]: pgmap v289: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:28.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:28 smithi167 ceph-mon[36229]: pgmap v289: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:28.582 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:28 smithi006 ceph-mon[31790]: pgmap v289: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:29.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:29 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:37:29.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:29 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:37:29.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:29 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:37:30.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:30 smithi006 ceph-mon[31790]: pgmap v290: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:30.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:30 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:37:30.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:30 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:37:30.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:30 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:37:30.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:30 smithi167 ceph-mon[36229]: pgmap v290: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:30.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:30 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:37:30.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:30 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:37:30.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:30 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:37:30.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:30 smithi090 ceph-mon[35681]: pgmap v290: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:30.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:30 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:37:30.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:30 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:37:30.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:30 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:37:32.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:32 smithi006 ceph-mon[31790]: pgmap v291: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:32.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:32 smithi167 ceph-mon[36229]: pgmap v291: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:32.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:32 smithi090 ceph-mon[35681]: pgmap v291: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:34.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:34 smithi006 ceph-mon[31790]: pgmap v292: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:34.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:34 smithi167 ceph-mon[36229]: pgmap v292: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:34.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:34 smithi090 ceph-mon[35681]: pgmap v292: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:36 smithi006 ceph-mon[31790]: pgmap v293: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:36.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:36 smithi167 ceph-mon[36229]: pgmap v293: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:36.617 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:36 smithi090 ceph-mon[35681]: pgmap v293: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:38.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:38 smithi006 ceph-mon[31790]: pgmap v294: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:38.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:38 smithi167 ceph-mon[36229]: pgmap v294: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:38.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:38 smithi090 ceph-mon[35681]: pgmap v294: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:40.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:40 smithi006 ceph-mon[31790]: pgmap v295: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:40.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:40 smithi167 ceph-mon[36229]: pgmap v295: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:40.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:40 smithi090 ceph-mon[35681]: pgmap v295: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:42.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:42 smithi006 ceph-mon[31790]: pgmap v296: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:42.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:42 smithi167 ceph-mon[36229]: pgmap v296: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:42.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:42 smithi090 ceph-mon[35681]: pgmap v296: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:44.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:44 smithi006 ceph-mon[31790]: pgmap v297: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:44 smithi167 ceph-mon[36229]: pgmap v297: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:44.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:44 smithi090 ceph-mon[35681]: pgmap v297: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:46.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:46 smithi006 ceph-mon[31790]: pgmap v298: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:46.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:46 smithi167 ceph-mon[36229]: pgmap v298: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:46.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:46 smithi090 ceph-mon[35681]: pgmap v298: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:48.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:48 smithi006 ceph-mon[31790]: pgmap v299: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:48.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:48 smithi167 ceph-mon[36229]: pgmap v299: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:48.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:48 smithi090 ceph-mon[35681]: pgmap v299: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:50.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:50 smithi006 ceph-mon[31790]: pgmap v300: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:50.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:50 smithi167 ceph-mon[36229]: pgmap v300: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:50.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:50 smithi090 ceph-mon[35681]: pgmap v300: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:52.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:52 smithi167 ceph-mon[36229]: pgmap v301: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:52.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:52 smithi090 ceph-mon[35681]: pgmap v301: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:52.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:52 smithi006 ceph-mon[31790]: pgmap v301: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:53.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:53 smithi167 ceph-mon[36229]: pgmap v302: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:53.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:53 smithi090 ceph-mon[35681]: pgmap v302: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:53.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:53 smithi006 ceph-mon[31790]: pgmap v302: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:55.864 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:55 smithi090 ceph-mon[35681]: pgmap v303: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:55.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:55 smithi006 ceph-mon[31790]: pgmap v303: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:55.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:55 smithi167 ceph-mon[36229]: pgmap v303: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:57.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:57 smithi090 ceph-mon[35681]: pgmap v304: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:57.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:57 smithi006 ceph-mon[31790]: pgmap v304: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:57.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:57 smithi167 ceph-mon[36229]: pgmap v304: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:37:59.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:37:59 smithi090 ceph-mon[35681]: pgmap v305: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:59.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:37:59 smithi006 ceph-mon[31790]: pgmap v305: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:37:59.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:37:59 smithi167 ceph-mon[36229]: pgmap v305: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:01.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:01 smithi090 ceph-mon[35681]: pgmap v306: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:01.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:01 smithi006 ceph-mon[31790]: pgmap v306: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:01.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:01 smithi167 ceph-mon[36229]: pgmap v306: 97 pgs: 97 active+clean; 76 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:03.865 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:03 smithi090 ceph-mon[35681]: pgmap v307: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:38:03.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:03 smithi006 ceph-mon[31790]: pgmap v307: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:38:03.994 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:03 smithi167 ceph-mon[36229]: pgmap v307: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-02-26T06:38:06.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:06 smithi090 ceph-mon[35681]: pgmap v308: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:38:06.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:06 smithi006 ceph-mon[31790]: pgmap v308: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:38:06.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:06 smithi167 ceph-mon[36229]: pgmap v308: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-02-26T06:38:07.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:07 smithi090 ceph-mon[35681]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T06:38:07.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:07 smithi006 ceph-mon[31790]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T06:38:07.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:07 smithi167 ceph-mon[36229]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T06:38:08.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:08 smithi090 ceph-mon[35681]: pgmap v310: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:38:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:08 smithi006 ceph-mon[31790]: pgmap v310: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:38:08.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:08 smithi167 ceph-mon[36229]: pgmap v310: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:38:09.156 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-26T06:38:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 -- 172.21.15.6:0/2427602015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 msgr2=0x7ff0e0110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 --2- 172.21.15.6:0/2427602015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e0110270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009890 tx=0x7ff0d802bb70 comp rx=0 tx=0).stop 2024-02-26T06:38:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 -- 172.21.15.6:0/2427602015 shutdown_connections 2024-02-26T06:38:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 --2- 172.21.15.6:0/2427602015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e0110270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 --2- 172.21.15.6:0/2427602015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0103f80 0x7ff0e0107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 --2- 172.21.15.6:0/2427602015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0e01035d0 0x7ff0e01039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 -- 172.21.15.6:0/2427602015 >> 172.21.15.6:0/2427602015 conn(0x7ff0e00fee70 msgr2=0x7ff0e0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 -- 172.21.15.6:0/2427602015 shutdown_connections 2024-02-26T06:38:09.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.379+0000 7ff0e6000640 1 -- 172.21.15.6:0/2427602015 wait complete. 2024-02-26T06:38:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.380+0000 7ff0e6000640 1 Processor -- start 2024-02-26T06:38:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 -- start start 2024-02-26T06:38:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0e01035d0 0x7ff0e019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 0x7ff0e0199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019a2e0 con 0x7ff0e0199830 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019a450 con 0x7ff0e01086a0 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0e6000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019a5c0 con 0x7ff0e01035d0 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0dfe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 0x7ff0e0199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0dea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0dfe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 0x7ff0e0199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49128/0 (socket says 172.21.15.6:49128) 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0dfe00640 1 -- 172.21.15.6:0/845525180 learned_addr learned my addr 172.21.15.6:0/845525180 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.381+0000 7ff0df400640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0e01035d0 0x7ff0e019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dfe00640 1 -- 172.21.15.6:0/845525180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0e01035d0 msgr2=0x7ff0e019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dfe00640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0e01035d0 0x7ff0e019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dfe00640 1 -- 172.21.15.6:0/845525180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 msgr2=0x7ff0e019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dfe00640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dfe00640 1 -- 172.21.15.6:0/845525180 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8009490 con 0x7ff0e0199830 2024-02-26T06:38:09.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dea00640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:09.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0dfe00640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 0x7ff0e0199ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff0d802c180 tx=0x7ff0d8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.382+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8034950 con 0x7ff0e0199830 2024-02-26T06:38:09.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.383+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0d802c9f0 con 0x7ff0e0199830 2024-02-26T06:38:09.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.383+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e01a6510 con 0x7ff0e0199830 2024-02-26T06:38:09.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.383+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8032ed0 con 0x7ff0e0199830 2024-02-26T06:38:09.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.383+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e01a6880 con 0x7ff0e0199830 2024-02-26T06:38:09.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.384+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e0104aa0 con 0x7ff0e0199830 2024-02-26T06:38:09.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.386+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff0d803b040 con 0x7ff0e0199830 2024-02-26T06:38:09.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.387+0000 7ff0d7e00640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff0c40a2220 0x7ff0c40a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.387+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8637+0+0 (secure 0 0 0) 0x7ff0d80efb60 con 0x7ff0e0199830 2024-02-26T06:38:09.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.388+0000 7ff0df400640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff0c40a2220 0x7ff0c40a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.388+0000 7ff0df400640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff0c40a2220 0x7ff0c40a46e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005fd0 tx=0x7ff0d0010040 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.393+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0d80b88e0 con 0x7ff0e0199830 2024-02-26T06:38:09.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.620+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 --> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0e0103f80 con 0x7ff0c40a2220 2024-02-26T06:38:09.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.633+0000 7ff0d7e00640 1 -- 172.21.15.6:0/845525180 <== mgr.15330 v2:172.21.15.6:6800/3331986569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7ff0e0103f80 con 0x7ff0c40a2220 2024-02-26T06:38:09.633 INFO:teuthology.orchestra.run.smithi006.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-26T00:00:00", "created": "2024-02-26T06:28:07", "first": "2024-02-26T06:29:00", "last": "2024-02-26T06:38:00", "last_pruned": "2024-02-26T06:38:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-02-26T06:38:09.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff0c40a2220 msgr2=0x7ff0c40a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff0c40a2220 0x7ff0c40a46e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005fd0 tx=0x7ff0d0010040 comp rx=0 tx=0).stop 2024-02-26T06:38:09.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 msgr2=0x7ff0e0199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 0x7ff0e0199ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff0d802c180 tx=0x7ff0d8004530 comp rx=0 tx=0).stop 2024-02-26T06:38:09.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 shutdown_connections 2024-02-26T06:38:09.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7ff0c40a2220 0x7ff0c40a46e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0e0199830 0x7ff0e0199ce0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e01086a0 0x7ff0e019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 --2- 172.21.15.6:0/845525180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0e01035d0 0x7ff0e019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 >> 172.21.15.6:0/845525180 conn(0x7ff0e00fee70 msgr2=0x7ff0e01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 shutdown_connections 2024-02-26T06:38:09.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.635+0000 7ff0e6000640 1 -- 172.21.15.6:0/845525180 wait complete. 2024-02-26T06:38:09.653 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-26T06:38:09.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.873+0000 7f700b000640 1 -- 172.21.15.6:0/4210604046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 msgr2=0x7f7004103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.873+0000 7f700b000640 1 --2- 172.21.15.6:0/4210604046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 0x7f7004103980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009930 tx=0x7f6ff402bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:09.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 -- 172.21.15.6:0/4210604046 shutdown_connections 2024-02-26T06:38:09.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 --2- 172.21.15.6:0/4210604046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f7004110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 --2- 172.21.15.6:0/4210604046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004103f50 0x7f7004107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 --2- 172.21.15.6:0/4210604046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 0x7f7004103980 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 -- 172.21.15.6:0/4210604046 >> 172.21.15.6:0/4210604046 conn(0x7f70040fede0 msgr2=0x7f7004101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:09.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 -- 172.21.15.6:0/4210604046 shutdown_connections 2024-02-26T06:38:09.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.874+0000 7f700b000640 1 -- 172.21.15.6:0/4210604046 wait complete. 2024-02-26T06:38:09.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 Processor -- start 2024-02-26T06:38:09.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 -- start start 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 0x7f700419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f700419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004199870 0x7f7004199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700419a260 con 0x7f70041035a0 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700419a3d0 con 0x7f7004108670 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.875+0000 7f700b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700419a590 con 0x7f7004199870 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7003400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f700419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7003400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f700419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48040/0 (socket says 172.21.15.6:48040) 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7003400640 1 -- 172.21.15.6:0/1174082014 learned_addr learned my addr 172.21.15.6:0/1174082014 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7009000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004199870 0x7f7004199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7003e00640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 0x7f700419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7009000640 1 -- 172.21.15.6:0/1174082014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 msgr2=0x7f700419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7009000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f700419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7009000640 1 -- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 msgr2=0x7f700419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7009000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 0x7f700419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7009000640 1 -- 172.21.15.6:0/1174082014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4009560 con 0x7f7004199870 2024-02-26T06:38:09.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.876+0000 7f7003400640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f700419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:09.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.877+0000 7f7009000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004199870 0x7f7004199d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009fd0 tx=0x7f6ffc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.877+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00ce20 con 0x7f7004199870 2024-02-26T06:38:09.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.877+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70041a6580 con 0x7f7004199870 2024-02-26T06:38:09.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.877+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc005020 con 0x7f7004199870 2024-02-26T06:38:09.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.877+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc010690 con 0x7f7004199870 2024-02-26T06:38:09.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.877+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70041a6a60 con 0x7f7004199870 2024-02-26T06:38:09.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.878+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700404f730 con 0x7f7004199870 2024-02-26T06:38:09.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.881+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f6ffc005500 con 0x7f7004199870 2024-02-26T06:38:09.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.882+0000 7f7000c00640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6fe80a2220 0x7f6fe80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:09.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.882+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8637+0+0 (secure 0 0 0) 0x7f6ffc014070 con 0x7f7004199870 2024-02-26T06:38:09.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.882+0000 7f7003e00640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6fe80a2220 0x7f6fe80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:09.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.883+0000 7f7003e00640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6fe80a2220 0x7f6fe80a46e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4005ab0 tx=0x7f6ff4005fd0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:09.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:09.884+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ffc097420 con 0x7f7004199870 2024-02-26T06:38:10.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.119+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 --> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f700410b930 con 0x7f6fe80a2220 2024-02-26T06:38:10.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.132+0000 7f7000c00640 1 -- 172.21.15.6:0/1174082014 <== mgr.15330 v2:172.21.15.6:6800/3331986569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f700410b930 con 0x7f6fe80a2220 2024-02-26T06:38:10.132 INFO:teuthology.orchestra.run.smithi006.stdout:/ 1m 6m3h 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.134+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6fe80a2220 msgr2=0x7f6fe80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.134+0000 7f700b000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6fe80a2220 0x7f6fe80a46e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4005ab0 tx=0x7f6ff4005fd0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.134+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004199870 msgr2=0x7f7004199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.134+0000 7f700b000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004199870 0x7f7004199d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009fd0 tx=0x7f6ffc00ef10 comp rx=0 tx=0).stop 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 shutdown_connections 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6fe80a2220 0x7f6fe80a46e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7004199870 0x7f7004199d20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004108670 0x7f700419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 --2- 172.21.15.6:0/1174082014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70041035a0 0x7f700419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 >> 172.21.15.6:0/1174082014 conn(0x7f70040fede0 msgr2=0x7f7004100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:10.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 shutdown_connections 2024-02-26T06:38:10.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.135+0000 7f700b000640 1 -- 172.21.15.6:0/1174082014 wait complete. 2024-02-26T06:38:10.154 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-26T06:38:10.165 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-26T06:38:10.165 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_vshankar_ceph_ba6fbbfee45488c6bcf0f2f12a3da33f007f0031/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-02-26T06:38:10.171 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-26T06:38:10.197 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:38:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:10 smithi090 ceph-mon[35681]: pgmap v311: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:38:10.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:10 smithi090 ceph-mon[35681]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T06:38:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:10 smithi006 ceph-mon[31790]: pgmap v311: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:38:10.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:10 smithi006 ceph-mon[31790]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T06:38:10.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 -- 172.21.15.6:0/2948485098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 msgr2=0x7f2e10073900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 --2- 172.21.15.6:0/2948485098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 0x7f2e10073900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009860 tx=0x7f2e0402bc10 comp rx=0 tx=0).stop 2024-02-26T06:38:10.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 -- 172.21.15.6:0/2948485098 shutdown_connections 2024-02-26T06:38:10.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 --2- 172.21.15.6:0/2948485098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e10112130 0x7f2e10114520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 --2- 172.21.15.6:0/2948485098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e10111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 --2- 172.21.15.6:0/2948485098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 0x7f2e10073900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.424+0000 7f2e17a00640 1 -- 172.21.15.6:0/2948485098 >> 172.21.15.6:0/2948485098 conn(0x7f2e100fcc30 msgr2=0x7f2e100ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:10.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.425+0000 7f2e17a00640 1 -- 172.21.15.6:0/2948485098 shutdown_connections 2024-02-26T06:38:10.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.425+0000 7f2e17a00640 1 -- 172.21.15.6:0/2948485098 wait complete. 2024-02-26T06:38:10.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 Processor -- start 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 -- start start 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 0x7f2e1019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e1019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e10112130 0x7f2e101a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101a2cd0 con 0x7f2e10073e40 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101a2e40 con 0x7f2e10112130 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e17a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101a3000 con 0x7f2e10073520 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e1019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.426+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e1019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49168/0 (socket says 172.21.15.6:49168) 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 -- 172.21.15.6:0/1318785437 learned_addr learned my addr 172.21.15.6:0/1318785437 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e15a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e10112130 0x7f2e101a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e15000640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 0x7f2e1019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 -- 172.21.15.6:0/1318785437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 msgr2=0x7f2e1019de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 0x7f2e1019de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 -- 172.21.15.6:0/1318785437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e10112130 msgr2=0x7f2e101a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e10112130 0x7f2e101a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 -- 172.21.15.6:0/1318785437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc009560 con 0x7f2e10073e40 2024-02-26T06:38:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.427+0000 7f2e0fe00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e1019e350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00e9c0 tx=0x7f2dfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.428+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00cde0 con 0x7f2e10073e40 2024-02-26T06:38:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.428+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2dfc0035f0 con 0x7f2e10073e40 2024-02-26T06:38:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.428+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e04009490 con 0x7f2e10073e40 2024-02-26T06:38:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.428+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc010670 con 0x7f2e10073e40 2024-02-26T06:38:10.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.428+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e101aa980 con 0x7f2e10073e40 2024-02-26T06:38:10.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.429+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e10068450 con 0x7f2e10073e40 2024-02-26T06:38:10.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.432+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f2dfc003ad0 con 0x7f2e10073e40 2024-02-26T06:38:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.433+0000 7f2e0d600640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f2df80a2230 0x7f2df80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.433+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2dfc014070 con 0x7f2e10073e40 2024-02-26T06:38:10.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.433+0000 7f2e15000640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f2df80a2230 0x7f2df80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.434+0000 7f2e15000640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f2df80a2230 0x7f2df80a46f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e0402c220 tx=0x7f2e040056d0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:10.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.438+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2dfc097160 con 0x7f2e10073e40 2024-02-26T06:38:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:10 smithi167 ceph-mon[36229]: pgmap v311: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-02-26T06:38:10.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:10 smithi167 ceph-mon[36229]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T06:38:10.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.641+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e1019f160 con 0x7f2e10073e40 2024-02-26T06:38:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.644+0000 7f2e0d600640 1 -- 172.21.15.6:0/1318785437 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+23131 (secure 0 0 0) 0x7f2dfc003760 con 0x7f2e10073e40 2024-02-26T06:38:10.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:38:10.645 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":96,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:38:10.069412+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1360414143":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/324687139":"2024-02-27T06:28:06.172431+0000","172.21.15.6:6801/324687139":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2856776212":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/3408505415":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/1854217832":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/64821628":"2024-02-27T06:28:06.172431+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:38:10.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f2df80a2230 msgr2=0x7f2df80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f2df80a2230 0x7f2df80a46f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e0402c220 tx=0x7f2e040056d0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 msgr2=0x7f2e1019e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e1019e350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00e9c0 tx=0x7f2dfc00ee80 comp rx=0 tx=0).stop 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 shutdown_connections 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f2df80a2230 0x7f2df80a46f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e10112130 0x7f2e101a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e10073e40 0x7f2e1019e350 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 --2- 172.21.15.6:0/1318785437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e10073520 0x7f2e1019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 >> 172.21.15.6:0/1318785437 conn(0x7f2e100fcc30 msgr2=0x7f2e100fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 shutdown_connections 2024-02-26T06:38:10.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.647+0000 7f2e17a00640 1 -- 172.21.15.6:0/1318785437 wait complete. 2024-02-26T06:38:10.668 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:38:10.669 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:38:10.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 -- 172.21.15.6:0/2363378397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1744102a30 msgr2=0x7f174410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 --2- 172.21.15.6:0/2363378397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1744102a30 0x7f174410af10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f172c009be0 tx=0x7f172c02bf80 comp rx=0 tx=0).stop 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 -- 172.21.15.6:0/2363378397 shutdown_connections 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 --2- 172.21.15.6:0/2363378397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174410b530 0x7f174410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 --2- 172.21.15.6:0/2363378397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1744102a30 0x7f174410af10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 --2- 172.21.15.6:0/2363378397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102110 0x7f17441024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 -- 172.21.15.6:0/2363378397 >> 172.21.15.6:0/2363378397 conn(0x7f1744076c10 msgr2=0x7f17440fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:10.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 -- 172.21.15.6:0/2363378397 shutdown_connections 2024-02-26T06:38:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.902+0000 7f1749400640 1 -- 172.21.15.6:0/2363378397 wait complete. 2024-02-26T06:38:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.903+0000 7f1749400640 1 Processor -- start 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 -- start start 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1744102110 0x7f174419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 0x7f174419f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174410b530 0x7f1744199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744199c70 con 0x7f1744102a30 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744199de0 con 0x7f174410b530 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1749400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744199fa0 con 0x7f1744102110 2024-02-26T06:38:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1743400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 0x7f174419f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1743400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 0x7f174419f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49200/0 (socket says 172.21.15.6:49200) 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1743400640 1 -- 172.21.15.6:0/1717742792 learned_addr learned my addr 172.21.15.6:0/1717742792 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1748a00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174410b530 0x7f1744199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.904+0000 7f1743e00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1744102110 0x7f174419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1748a00640 1 -- 172.21.15.6:0/1717742792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1744102110 msgr2=0x7f174419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1748a00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1744102110 0x7f174419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1748a00640 1 -- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 msgr2=0x7f174419f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1748a00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 0x7f174419f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1748a00640 1 -- 172.21.15.6:0/1717742792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c009840 con 0x7f174410b530 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1743400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 0x7f174419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1748a00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174410b530 0x7f1744199620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f173c00d6b0 tx=0x7f173c00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c004220 con 0x7f174410b530 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f173c004380 con 0x7f174410b530 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0099b0 con 0x7f174410b530 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.905+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174419a280 con 0x7f174410b530 2024-02-26T06:38:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.906+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17441a6510 con 0x7f174410b530 2024-02-26T06:38:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.907+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174404eee0 con 0x7f174410b530 2024-02-26T06:38:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.910+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f173c0046c0 con 0x7f174410b530 2024-02-26T06:38:10.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.911+0000 7f1740c00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f17240a22b0 0x7f17240a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:10.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.911+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f173c0cdf60 con 0x7f174410b530 2024-02-26T06:38:10.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.911+0000 7f1743e00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f17240a22b0 0x7f17240a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:10.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.912+0000 7f1743e00640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f17240a22b0 0x7f17240a4770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1734005fd0 tx=0x7f1734009260 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:10.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:10.913+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f173c096bc0 con 0x7f174410b530 2024-02-26T06:38:11.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.115+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1744111110 con 0x7f174410b530 2024-02-26T06:38:11.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.117+0000 7f1740c00640 1 -- 172.21.15.6:0/1717742792 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f173c017070 con 0x7f174410b530 2024-02-26T06:38:11.117 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:38:11.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.119+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f17240a22b0 msgr2=0x7f17240a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f17240a22b0 0x7f17240a4770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1734005fd0 tx=0x7f1734009260 comp rx=0 tx=0).stop 2024-02-26T06:38:11.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174410b530 msgr2=0x7f1744199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174410b530 0x7f1744199620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f173c00d6b0 tx=0x7f173c00dab0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 shutdown_connections 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f17240a22b0 0x7f17240a4770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174410b530 0x7f1744199620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1744102a30 0x7f174419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 --2- 172.21.15.6:0/1717742792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1744102110 0x7f174419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 >> 172.21.15.6:0/1717742792 conn(0x7f1744076c10 msgr2=0x7f17440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 shutdown_connections 2024-02-26T06:38:11.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.120+0000 7f1749400640 1 -- 172.21.15.6:0/1717742792 wait complete. 2024-02-26T06:38:11.140 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:38:11.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:11 smithi090 ceph-mon[35681]: from='client.15954 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:38:11.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:11 smithi090 ceph-mon[35681]: from='client.26086 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:38:11.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:11 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1318785437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:11.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 -- 172.21.15.6:0/372251575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 msgr2=0x7f1248103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 --2- 172.21.15.6:0/372251575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f1248103790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1238009930 tx=0x7f123802bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:11.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 -- 172.21.15.6:0/372251575 shutdown_connections 2024-02-26T06:38:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 --2- 172.21.15.6:0/372251575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 0x7f1248110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 --2- 172.21.15.6:0/372251575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248103d60 0x7f1248107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 --2- 172.21.15.6:0/372251575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f1248103790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 -- 172.21.15.6:0/372251575 >> 172.21.15.6:0/372251575 conn(0x7f12480fec30 msgr2=0x7f1248101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 -- 172.21.15.6:0/372251575 shutdown_connections 2024-02-26T06:38:11.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.371+0000 7f1247e00640 1 -- 172.21.15.6:0/372251575 wait complete. 2024-02-26T06:38:11.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.372+0000 7f1247e00640 1 Processor -- start 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 -- start start 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f124819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 0x7f124819fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248199680 0x7f1248199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819a150 con 0x7f1248108480 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819a2c0 con 0x7f12481033b0 2024-02-26T06:38:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819a480 con 0x7f1248199680 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1246000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 0x7f124819fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1246a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f124819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1247400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248199680 0x7f1248199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1246000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 0x7f124819fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49218/0 (socket says 172.21.15.6:49218) 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246000640 1 -- 172.21.15.6:0/1571639445 learned_addr learned my addr 172.21.15.6:0/1571639445 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.373+0000 7f1246a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f124819f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48094/0 (socket says 172.21.15.6:48094) 2024-02-26T06:38:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246a00640 1 -- 172.21.15.6:0/1571639445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248199680 msgr2=0x7f1248199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246a00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248199680 0x7f1248199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246a00640 1 -- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 msgr2=0x7f124819fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246a00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 0x7f124819fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246a00640 1 -- 172.21.15.6:0/1571639445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238009560 con 0x7f12481033b0 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1247400640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248199680 0x7f1248199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.374+0000 7f1246a00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f124819f5c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1238009230 tx=0x7f123802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.375+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12380042d0 con 0x7f12481033b0 2024-02-26T06:38:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.375+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1238004430 con 0x7f12481033b0 2024-02-26T06:38:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.375+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238003bd0 con 0x7f12481033b0 2024-02-26T06:38:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.375+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12481a63c0 con 0x7f12481033b0 2024-02-26T06:38:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.375+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12481a6850 con 0x7f12481033b0 2024-02-26T06:38:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.376+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1248104880 con 0x7f12481033b0 2024-02-26T06:38:11.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.378+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f1238003d30 con 0x7f12481033b0 2024-02-26T06:38:11.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.379+0000 7f123f400640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f122c0a2230 0x7f122c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.379+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f12380eff40 con 0x7f12481033b0 2024-02-26T06:38:11.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.380+0000 7f1246000640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f122c0a2230 0x7f122c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.380+0000 7f1246000640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f122c0a2230 0x7f122c0a46f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f124819aed0 tx=0x7f1234005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.384+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12380b8c30 con 0x7f12481033b0 2024-02-26T06:38:11.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:11 smithi006 ceph-mon[31790]: from='client.15954 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:38:11.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:11 smithi006 ceph-mon[31790]: from='client.26086 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:38:11.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:11 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1318785437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:11.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:11 smithi167 ceph-mon[36229]: from='client.15954 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:38:11.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:11 smithi167 ceph-mon[36229]: from='client.26086 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T06:38:11.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:11 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1318785437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:11.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.581+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f124819abb0 con 0x7f12481033b0 2024-02-26T06:38:11.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.582+0000 7f123f400640 1 -- 172.21.15.6:0/1571639445 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12380bc2f0 con 0x7f12481033b0 2024-02-26T06:38:11.582 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:38:11.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f122c0a2230 msgr2=0x7f122c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f122c0a2230 0x7f122c0a46f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f124819aed0 tx=0x7f1234005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 msgr2=0x7f124819f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f124819f5c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1238009230 tx=0x7f123802c5d0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 shutdown_connections 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f122c0a2230 0x7f122c0a46f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248199680 0x7f1248199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248108480 0x7f124819fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 --2- 172.21.15.6:0/1571639445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12481033b0 0x7f124819f5c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 >> 172.21.15.6:0/1571639445 conn(0x7f12480fec30 msgr2=0x7f1248100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:11.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 shutdown_connections 2024-02-26T06:38:11.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.585+0000 7f1247e00640 1 -- 172.21.15.6:0/1571639445 wait complete. 2024-02-26T06:38:11.603 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:38:11.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 -- 172.21.15.6:0/3703233812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102d40 msgr2=0x7f2634107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 --2- 172.21.15.6:0/3703233812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102d40 0x7f2634107170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2620009930 tx=0x7f262002bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 -- 172.21.15.6:0/3703233812 shutdown_connections 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 --2- 172.21.15.6:0/3703233812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2634107880 0x7f263410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 --2- 172.21.15.6:0/3703233812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102d40 0x7f2634107170 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 --2- 172.21.15.6:0/3703233812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2634102390 0x7f2634102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 -- 172.21.15.6:0/3703233812 >> 172.21.15.6:0/3703233812 conn(0x7f2634077260 msgr2=0x7f2634077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 -- 172.21.15.6:0/3703233812 shutdown_connections 2024-02-26T06:38:11.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.829+0000 7f263ba00640 1 -- 172.21.15.6:0/3703233812 wait complete. 2024-02-26T06:38:11.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.830+0000 7f263ba00640 1 Processor -- start 2024-02-26T06:38:11.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.830+0000 7f263ba00640 1 -- start start 2024-02-26T06:38:11.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.830+0000 7f263ba00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 0x7f263419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f263ba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2634107880 0x7f263419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f263ba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26341979a0 0x7f2634197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f263ba00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634198390 con 0x7f2634102390 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f263ba00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634198500 con 0x7f2634107880 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f263ba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341986c0 con 0x7f26341979a0 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 0x7f263419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 0x7f263419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49226/0 (socket says 172.21.15.6:49226) 2024-02-26T06:38:11.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 -- 172.21.15.6:0/265746570 learned_addr learned my addr 172.21.15.6:0/265746570 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2633e00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2634107880 0x7f263419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639a00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26341979a0 0x7f2634197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 -- 172.21.15.6:0/265746570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26341979a0 msgr2=0x7f2634197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26341979a0 0x7f2634197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 -- 172.21.15.6:0/265746570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2634107880 msgr2=0x7f263419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2634107880 0x7f263419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.831+0000 7f2639000640 1 -- 172.21.15.6:0/265746570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c009840 con 0x7f2634102390 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.832+0000 7f2639000640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 0x7f263419d7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f262800e9f0 tx=0x7f262800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:11.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.832+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628009700 con 0x7f2634102390 2024-02-26T06:38:11.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.832+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f262800cd80 con 0x7f2634102390 2024-02-26T06:38:11.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.832+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2620009560 con 0x7f2634102390 2024-02-26T06:38:11.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.832+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628018920 con 0x7f2634102390 2024-02-26T06:38:11.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.832+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26341a4a00 con 0x7f2634102390 2024-02-26T06:38:11.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.833+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2634068450 con 0x7f2634102390 2024-02-26T06:38:11.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.835+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f2628018a80 con 0x7f2634102390 2024-02-26T06:38:11.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.837+0000 7f2631600640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f261c0a22b0 0x7f261c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:11.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.837+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f2628014070 con 0x7f2634102390 2024-02-26T06:38:11.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.837+0000 7f2633e00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f261c0a22b0 0x7f261c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:11.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.838+0000 7f2633e00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f261c0a22b0 0x7f261c0a4770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2620008000 tx=0x7f2620000bc0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:11.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:11.839+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2628097560 con 0x7f2634102390 2024-02-26T06:38:12.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.041+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2634109890 con 0x7f2634102390 2024-02-26T06:38:12.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.042+0000 7f2631600640 1 -- 172.21.15.6:0/265746570 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26280260a0 con 0x7f2634102390 2024-02-26T06:38:12.043 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:38:12.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f261c0a22b0 msgr2=0x7f261c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:12.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f261c0a22b0 0x7f261c0a4770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2620008000 tx=0x7f2620000bc0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 msgr2=0x7f263419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:12.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 0x7f263419d7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f262800e9f0 tx=0x7f262800eeb0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 shutdown_connections 2024-02-26T06:38:12.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f261c0a22b0 0x7f261c0a4770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26341979a0 0x7f2634197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2634107880 0x7f263419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 --2- 172.21.15.6:0/265746570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2634102390 0x7f263419d7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 >> 172.21.15.6:0/265746570 conn(0x7f2634077260 msgr2=0x7f26340fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:12.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 shutdown_connections 2024-02-26T06:38:12.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.045+0000 7f263ba00640 1 -- 172.21.15.6:0/265746570 wait complete. 2024-02-26T06:38:12.065 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-26T06:38:12.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.291+0000 7f4e17200640 1 -- 172.21.15.6:0/1969310380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e181075c0 msgr2=0x7f4e18107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:12.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.291+0000 7f4e17200640 1 --2- 172.21.15.6:0/1969310380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e181075c0 0x7f4e18107a40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009900 tx=0x7f4e0402bcd0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 -- 172.21.15.6:0/1969310380 shutdown_connections 2024-02-26T06:38:12.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 --2- 172.21.15.6:0/1969310380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e18107f80 0x7f4e18114420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 --2- 172.21.15.6:0/1969310380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e181075c0 0x7f4e18107a40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 --2- 172.21.15.6:0/1969310380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 -- 172.21.15.6:0/1969310380 >> 172.21.15.6:0/1969310380 conn(0x7f4e18076d20 msgr2=0x7f4e18079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:12.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 -- 172.21.15.6:0/1969310380 shutdown_connections 2024-02-26T06:38:12.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.292+0000 7f4e17200640 1 -- 172.21.15.6:0/1969310380 wait complete. 2024-02-26T06:38:12.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.293+0000 7f4e17200640 1 Processor -- start 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.293+0000 7f4e17200640 1 -- start start 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e17200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 0x7f4e1819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e17200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e17200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e181a2690 0x7f4e181a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e17200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181a3080 con 0x7f4e18107f80 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e17200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181a31f0 con 0x7f4e1810c530 2024-02-26T06:38:12.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e17200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181a3360 con 0x7f4e181a2690 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 0x7f4e1819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 0x7f4e1819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49248/0 (socket says 172.21.15.6:49248) 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15e00640 1 -- 172.21.15.6:0/1917028729 learned_addr learned my addr 172.21.15.6:0/1917028729 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:48122/0 (socket says 172.21.15.6:48122) 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e16800640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e181a2690 0x7f4e181a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 -- 172.21.15.6:0/1917028729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e181a2690 msgr2=0x7f4e181a2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:12.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e181a2690 0x7f4e181a2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 -- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 msgr2=0x7f4e1819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 0x7f4e1819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15400640 1 -- 172.21.15.6:0/1917028729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009560 con 0x7f4e1810c530 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e15e00640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 0x7f4e1819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.294+0000 7f4e16800640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e181a2690 0x7f4e181a2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.295+0000 7f4e15400640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1819e310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4e04008000 tx=0x7f4e0402c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.295+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0402db20 con 0x7f4e1810c530 2024-02-26T06:38:12.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.295+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e181aa940 con 0x7f4e1810c530 2024-02-26T06:38:12.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.295+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e0402dc80 con 0x7f4e1810c530 2024-02-26T06:38:12.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.295+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04003a10 con 0x7f4e1810c530 2024-02-26T06:38:12.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.295+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e181aad60 con 0x7f4e1810c530 2024-02-26T06:38:12.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.296+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1804eee0 con 0x7f4e1810c530 2024-02-26T06:38:12.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.298+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4e04005360 con 0x7f4e1810c530 2024-02-26T06:38:12.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.300+0000 7f4e0ea00640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4dfc0a2290 0x7f4dfc0a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:12.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.300+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f4e040f0470 con 0x7f4e1810c530 2024-02-26T06:38:12.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.300+0000 7f4e15e00640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4dfc0a2290 0x7f4dfc0a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:12.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.301+0000 7f4e15e00640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4dfc0a2290 0x7f4dfc0a4750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e080058c0 tx=0x7f4e08005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:12.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.304+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e040b9160 con 0x7f4e1810c530 2024-02-26T06:38:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:12 smithi090 ceph-mon[35681]: pgmap v313: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:38:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:12 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1717742792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:12 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1571639445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:12 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/265746570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:12 smithi006 ceph-mon[31790]: pgmap v313: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:38:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:12 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1717742792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:12 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1571639445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:12 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/265746570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:12 smithi167 ceph-mon[36229]: pgmap v313: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-02-26T06:38:12.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:12 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1717742792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:12 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1571639445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:12 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/265746570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:38:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.558+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f4e181176f0 con 0x7f4e1810c530 2024-02-26T06:38:12.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.571+0000 7f4e0ea00640 1 -- 172.21.15.6:0/1917028729 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) v1 ==== 81+0+736935 (secure 0 0 0) 0x7f4e040bc820 con 0x7f4e1810c530 2024-02-26T06:38:12.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:38:12.584 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:38:12.584 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 28, 2024-02-26T06:38:12.584 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "active_gid": 15330, 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "active_name": "x", 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addrs": { 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6800", 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3331986569 2024-02-26T06:38:12.585 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6801", 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3331986569 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.586 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addr": "172.21.15.6:6801/3331986569", 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "active_change": "2024-02-26T06:28:06.172909+0000", 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "active_mgr_features": 4540701547738038271, 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "available": true, 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "standbys": [ 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 25438, 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "z", 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540701547738038271, 2024-02-26T06:38:12.587 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:12.588 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T06:38:12.589 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.591 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.592 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.593 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.595 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.596 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.597 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.598 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.599 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-02-26T06:38:12.600 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.601 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.602 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.603 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.605 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-02-26T06:38:12.606 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.607 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.611 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.612 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.614 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-02-26T06:38:12.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T06:38:12.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.617 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-02-26T06:38:12.620 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-02-26T06:38:12.621 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.622 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.623 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-02-26T06:38:12.624 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.625 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.626 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.629 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.630 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.633 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.634 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.635 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-02-26T06:38:12.636 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-02-26T06:38:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-02-26T06:38:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-02-26T06:38:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T06:38:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.649 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-02-26T06:38:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:12.653 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.658 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-02-26T06:38:12.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.663 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T06:38:12.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.668 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T06:38:12.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-02-26T06:38:12.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.676 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T06:38:12.677 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-02-26T06:38:12.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:12.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-02-26T06:38:12.683 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-02-26T06:38:12.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.686 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.687 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.688 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.689 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.690 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.692 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-02-26T06:38:12.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.694 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.695 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.698 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.699 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.700 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.701 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.702 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.704 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T06:38:12.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.710 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.715 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-02-26T06:38:12.716 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-02-26T06:38:12.717 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.718 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-02-26T06:38:12.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-02-26T06:38:12.722 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-02-26T06:38:12.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T06:38:12.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.727 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-02-26T06:38:12.728 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T06:38:12.729 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.730 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.731 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.732 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-02-26T06:38:12.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T06:38:12.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.737 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-02-26T06:38:12.738 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T06:38:12.739 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.740 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.741 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-02-26T06:38:12.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.744 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.756 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-02-26T06:38:12.757 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.758 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.760 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T06:38:12.761 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.764 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T06:38:12.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.767 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.768 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.769 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T06:38:12.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T06:38:12.776 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.777 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.778 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.779 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.780 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T06:38:12.781 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.782 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-02-26T06:38:12.786 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.788 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.789 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.790 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.791 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T06:38:12.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T06:38:12.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.801 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.802 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T06:38:12.804 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.805 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.806 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.808 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.809 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.810 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.811 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.812 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.814 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-02-26T06:38:12.816 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T06:38:12.817 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.819 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.820 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:12.821 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T06:38:12.822 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.823 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.824 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.825 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:12.826 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T06:38:12.827 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.828 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.829 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.830 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.831 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-02-26T06:38:12.832 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.833 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.834 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.835 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-02-26T06:38:12.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-02-26T06:38:12.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.838 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.839 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.840 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.842 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.843 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.844 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.845 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.846 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-02-26T06:38:12.847 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.848 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.849 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:12.852 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.853 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.854 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.856 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-02-26T06:38:12.857 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.858 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-02-26T06:38:12.859 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.862 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.863 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.864 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.866 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.867 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.868 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.869 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.870 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.871 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.872 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-02-26T06:38:12.873 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T06:38:12.874 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.875 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.876 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.877 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-02-26T06:38:12.878 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T06:38:12.879 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.880 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.881 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.882 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-02-26T06:38:12.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T06:38:12.884 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.885 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.887 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.888 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.889 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.891 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.892 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.893 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-02-26T06:38:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-02-26T06:38:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-02-26T06:38:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.911 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.912 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.913 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.914 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.915 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.916 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.918 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-02-26T06:38:12.919 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T06:38:12.920 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.921 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.922 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.923 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.924 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.925 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.926 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.927 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.928 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.930 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.931 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.933 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.934 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.935 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.936 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.937 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.938 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.939 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.940 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.941 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.942 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.943 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.944 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:12.945 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.946 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.947 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.948 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.949 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.950 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.951 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.952 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.953 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.954 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.955 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.956 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.957 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.958 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.959 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.960 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.961 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.962 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.963 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.964 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.965 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.966 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.967 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-02-26T06:38:12.968 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.969 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.970 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.971 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-02-26T06:38:12.972 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T06:38:12.973 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.974 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.975 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.976 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.977 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.978 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.980 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.981 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.982 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.983 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.984 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.985 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.986 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.987 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.988 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.989 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:12.990 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.991 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-02-26T06:38:12.992 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.994 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:12.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.996 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:12.997 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:12.998 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:12.999 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.000 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.002 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.003 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.004 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.005 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.006 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.007 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.008 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.009 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.010 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.011 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.012 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.013 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.014 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.015 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.016 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.017 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.018 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.019 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.020 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.021 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.022 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.023 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.026 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.027 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.028 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.029 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.030 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.031 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.033 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.034 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.035 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.036 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.037 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.038 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.039 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.040 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.041 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.042 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-02-26T06:38:13.043 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.044 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.045 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.046 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.047 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.048 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.049 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.050 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.051 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.052 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.053 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.054 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.055 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.056 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.057 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.058 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.059 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.060 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.061 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-02-26T06:38:13.062 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T06:38:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.064 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.065 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.066 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.067 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.068 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.069 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.070 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.071 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-02-26T06:38:13.072 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.073 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.074 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.075 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.076 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:13.077 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T06:38:13.078 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.079 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.080 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.081 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-02-26T06:38:13.082 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-02-26T06:38:13.083 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-02-26T06:38:13.084 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.085 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.086 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.087 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.088 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.089 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.090 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.091 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.092 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.093 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.095 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.096 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.098 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.099 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.100 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.101 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-02-26T06:38:13.102 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.103 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.104 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.105 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.106 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.107 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.108 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.109 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.110 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.111 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:13.112 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.113 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.114 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.116 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.118 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.119 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.120 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.121 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.122 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.123 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.124 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: "drive_group_interval": { 2024-02-26T06:38:13.125 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "drive_group_interval", 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300.0", 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.126 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.127 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.128 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.129 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.130 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.131 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.133 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.134 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-02-26T06:38:13.135 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T06:38:13.136 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.137 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.138 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.139 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.140 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.141 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.144 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.145 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.146 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.147 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-02-26T06:38:13.148 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.149 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.150 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.151 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.152 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.153 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.154 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.155 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.156 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.157 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.158 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-02-26T06:38:13.159 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.160 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.161 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-02-26T06:38:13.162 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.163 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.164 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.165 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-02-26T06:38:13.166 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T06:38:13.167 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.168 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.169 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.170 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.171 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.173 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.174 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.175 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.176 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.177 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.178 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.179 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.181 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.182 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.183 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.184 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.186 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.187 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.188 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.189 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.190 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.192 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.193 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.194 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.195 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-02-26T06:38:13.196 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.197 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.198 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.199 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.200 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.201 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.202 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.203 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.204 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.205 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.206 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.207 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-02-26T06:38:13.208 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.209 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-02-26T06:38:13.210 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.211 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.212 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.213 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-02-26T06:38:13.215 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.216 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.217 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.218 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-02-26T06:38:13.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-02-26T06:38:13.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-02-26T06:38:13.225 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.226 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.227 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.229 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.230 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.231 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.234 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.235 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.236 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.237 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-02-26T06:38:13.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.239 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.240 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.241 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.242 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.248 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.249 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.250 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.252 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.254 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.255 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.256 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.447 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.447 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.449 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.450 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.451 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.453 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.453 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.453 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.453 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.453 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.453 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.454 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-02-26T06:38:13.454 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-02-26T06:38:13.454 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.454 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.454 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.455 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.456 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.457 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.458 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.459 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-02-26T06:38:13.460 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.461 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.462 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.463 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.464 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.465 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.466 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.467 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.468 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.469 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.469 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.469 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.469 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.469 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.469 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.470 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.470 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.470 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.471 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.472 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.473 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.474 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.474 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.474 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.474 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.475 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.475 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.475 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.475 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.475 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.476 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.476 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.476 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.476 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.476 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-02-26T06:38:13.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.478 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.479 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T06:38:13.480 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.481 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 25454, 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "y", 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540701547738038271, 2024-02-26T06:38:13.482 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.483 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T06:38:13.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.485 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.486 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.487 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.489 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.490 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.491 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.492 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.493 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.494 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.495 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.496 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-02-26T06:38:13.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-02-26T06:38:13.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-02-26T06:38:13.497 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.498 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.499 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.500 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-02-26T06:38:13.501 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.502 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.503 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.504 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-02-26T06:38:13.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-02-26T06:38:13.507 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-02-26T06:38:13.508 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.509 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.510 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.511 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:13.512 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.513 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:13.514 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-02-26T06:38:13.515 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T06:38:13.516 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.517 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.518 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.519 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-02-26T06:38:13.520 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:38:13.521 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.522 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.523 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.524 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.527 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.529 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.530 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.531 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-02-26T06:38:13.533 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.534 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.535 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-02-26T06:38:13.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T06:38:13.538 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.540 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-02-26T06:38:13.541 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T06:38:13.542 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.543 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.544 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.545 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.546 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.547 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:13.548 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-02-26T06:38:13.549 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.550 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.551 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.552 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.553 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.555 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-02-26T06:38:13.556 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.557 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.558 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-02-26T06:38:13.561 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.562 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T06:38:13.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.564 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.565 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.566 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.568 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-02-26T06:38:13.569 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-02-26T06:38:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.571 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.572 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.573 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.574 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.575 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T06:38:13.576 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-02-26T06:38:13.577 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:13.578 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.579 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.580 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.581 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-02-26T06:38:13.582 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.583 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:13.584 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-02-26T06:38:13.585 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T06:38:13.586 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.587 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.588 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.589 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-02-26T06:38:13.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T06:38:13.591 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.592 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.593 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.594 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.595 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.596 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.597 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.599 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.601 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T06:38:13.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.604 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.605 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.607 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-02-26T06:38:13.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-02-26T06:38:13.612 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.613 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.614 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.615 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.616 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.617 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.618 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.619 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T06:38:13.620 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-02-26T06:38:13.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T06:38:13.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-02-26T06:38:13.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-02-26T06:38:13.630 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.631 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.632 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.633 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-02-26T06:38:13.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T06:38:13.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.640 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.642 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.646 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.647 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.648 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-02-26T06:38:13.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.652 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.653 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T06:38:13.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.661 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.663 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.665 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.666 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T06:38:13.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.671 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:13.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T06:38:13.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.680 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.681 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T06:38:13.682 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.685 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.686 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.688 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T06:38:13.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.692 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.696 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T06:38:13.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.699 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.702 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.704 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T06:38:13.705 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T06:38:13.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.707 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.712 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.714 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.715 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.716 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.717 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.718 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T06:38:13.719 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.720 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.721 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.722 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.723 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.726 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-02-26T06:38:13.727 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.728 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.729 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.730 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.731 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.732 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.733 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.734 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.735 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.740 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.741 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-02-26T06:38:13.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:13.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.747 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-02-26T06:38:13.748 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-02-26T06:38:13.749 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.750 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.751 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.752 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-02-26T06:38:13.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.755 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.756 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.757 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.758 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.759 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.760 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.761 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.762 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.766 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-02-26T06:38:13.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-02-26T06:38:13.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T06:38:13.771 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.772 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.775 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.776 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.777 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.781 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-02-26T06:38:13.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.785 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.786 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.787 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:13.788 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.789 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-02-26T06:38:13.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T06:38:13.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.792 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.793 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.794 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:13.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T06:38:13.796 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.797 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.798 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.801 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.802 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.803 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.804 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-02-26T06:38:13.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.806 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-02-26T06:38:13.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.808 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.812 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-02-26T06:38:13.813 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.814 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.816 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.817 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.818 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.819 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.820 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.821 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.822 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.823 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-02-26T06:38:13.824 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.825 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.826 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.827 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.828 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.829 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.830 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.832 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-02-26T06:38:13.833 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-02-26T06:38:13.834 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-02-26T06:38:13.835 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.836 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.837 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.838 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.839 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.840 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.841 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.842 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.843 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.844 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.846 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.847 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.848 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.849 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.850 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.851 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.852 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.853 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.855 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.856 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.857 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-02-26T06:38:13.858 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.859 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-02-26T06:38:13.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.866 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.867 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.868 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.869 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.870 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.871 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.874 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.875 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.876 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.877 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.878 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.879 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.882 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.883 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.884 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.885 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.886 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.888 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.889 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.890 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.891 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.892 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.893 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.894 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.895 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.896 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.897 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.898 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.899 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.900 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.901 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.902 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.903 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.904 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.905 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.906 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.907 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.908 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.909 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.910 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.912 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.913 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.914 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.915 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.916 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.918 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.919 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.920 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.921 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.922 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.923 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.924 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.925 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.926 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.927 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.928 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.929 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.930 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.931 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.932 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.933 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.934 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.935 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-02-26T06:38:13.936 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.937 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.938 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.939 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.940 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.941 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.942 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.943 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.944 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.945 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.946 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.947 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.948 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.949 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.950 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.951 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.952 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.953 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.954 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.955 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.956 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.958 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.959 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.960 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.961 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.962 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T06:38:13.963 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.964 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.965 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.966 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.967 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.968 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.969 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-02-26T06:38:13.970 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-02-26T06:38:13.971 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.972 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-02-26T06:38:13.973 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.974 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-02-26T06:38:13.975 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.976 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.977 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.978 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.979 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.980 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.981 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:13.982 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.983 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.984 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.985 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.986 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-02-26T06:38:13.987 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.988 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:13.989 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.990 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.991 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.992 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:13.993 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.994 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.995 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.996 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:13.997 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:13.998 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:13.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:14.001 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.002 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.003 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.004 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.005 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.006 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.007 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.008 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.009 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.010 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.011 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.013 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.014 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "drive_group_interval": { 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "drive_group_interval", 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300.0", 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.015 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.016 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.017 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.018 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.019 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.020 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.021 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.022 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.023 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.024 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.025 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.026 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.027 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.028 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.029 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.030 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.031 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.032 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.033 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.034 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.035 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.036 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-02-26T06:38:14.037 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-02-26T06:38:14.038 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.039 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.040 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.041 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.042 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-02-26T06:38:14.043 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.044 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.045 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.046 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.047 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-02-26T06:38:14.048 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.049 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.050 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.051 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.052 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-02-26T06:38:14.053 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.054 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.055 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.056 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-02-26T06:38:14.057 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-02-26T06:38:14.058 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.059 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.060 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.061 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.062 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.063 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.064 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.065 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.066 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.067 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.068 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.069 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.070 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.071 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.072 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.073 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.074 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.075 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.076 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.077 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.078 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.079 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.080 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.081 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-02-26T06:38:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T06:38:14.099 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.100 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.101 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.102 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-02-26T06:38:14.103 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T06:38:14.104 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.105 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.106 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.107 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-02-26T06:38:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T06:38:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.111 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.112 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.113 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-02-26T06:38:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.117 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.118 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.119 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.120 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.121 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.122 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.123 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.124 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.125 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.126 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.127 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.128 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-02-26T06:38:14.129 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.130 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.131 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.133 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.134 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.135 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.136 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.137 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.138 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.139 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.140 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.141 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.143 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.144 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.145 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.146 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.147 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.148 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.149 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.150 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.151 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.153 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:14.154 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.155 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:14.156 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.157 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.158 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.159 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.160 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.161 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.162 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.163 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.164 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.165 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.166 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.167 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.168 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-02-26T06:38:14.169 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.170 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.171 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.172 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T06:38:14.173 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.174 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: "modules": [ 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard", 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: "iostat", 2024-02-26T06:38:14.175 INFO:teuthology.orchestra.run.smithi006.stdout: "nfs", 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "restful", 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "snap_schedule" 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.176 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.177 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.178 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.179 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.180 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.181 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.182 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.183 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.184 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.185 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.186 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-02-26T06:38:14.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-02-26T06:38:14.187 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.187 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.187 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.187 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.188 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.188 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.188 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-02-26T06:38:14.188 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.189 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.189 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.189 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-02-26T06:38:14.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-02-26T06:38:14.190 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.190 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.190 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-02-26T06:38:14.191 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.191 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.191 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.191 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-02-26T06:38:14.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.193 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.194 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.195 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.196 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.197 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-02-26T06:38:14.198 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T06:38:14.199 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.200 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.201 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-02-26T06:38:14.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T06:38:14.203 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.204 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-02-26T06:38:14.206 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-02-26T06:38:14.207 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.208 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.209 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.210 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.211 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.212 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.213 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-02-26T06:38:14.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.215 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.216 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.218 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.219 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.220 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.221 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-02-26T06:38:14.225 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-02-26T06:38:14.226 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.228 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:14.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.231 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.232 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.233 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.234 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.235 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.237 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.238 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.239 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.240 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.241 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.242 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.243 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.244 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.245 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.246 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.247 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.248 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.249 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.250 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.251 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.252 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-02-26T06:38:14.253 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.254 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.255 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.257 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-02-26T06:38:14.258 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.259 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.260 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.261 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.262 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.263 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.264 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T06:38:14.265 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.266 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.267 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.268 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.269 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T06:38:14.270 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.271 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.272 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.273 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.274 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.275 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-02-26T06:38:14.276 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T06:38:14.277 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.278 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.279 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-02-26T06:38:14.281 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T06:38:14.282 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.283 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.284 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.285 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-02-26T06:38:14.286 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.287 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.289 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.290 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.291 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.292 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.297 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.298 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.299 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.300 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.301 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T06:38:14.302 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T06:38:14.303 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.304 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.305 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-02-26T06:38:14.307 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T06:38:14.308 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.309 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.310 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.311 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.312 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-02-26T06:38:14.313 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-02-26T06:38:14.314 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T06:38:14.315 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.316 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.317 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-02-26T06:38:14.319 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.320 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.321 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.322 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.323 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.324 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-02-26T06:38:14.327 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T06:38:14.328 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.329 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.330 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.331 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.332 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.333 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.334 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.336 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.337 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.338 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.339 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.340 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.341 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.342 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.343 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.344 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.345 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.346 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.347 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.348 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.350 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.351 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.352 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-02-26T06:38:14.353 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.354 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.355 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.356 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.357 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.358 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.359 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.360 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.361 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.362 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.363 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.365 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T06:38:14.366 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.367 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.368 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.369 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.370 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T06:38:14.371 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.373 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.374 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.375 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.376 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.377 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.378 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.379 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.380 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.381 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.382 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.383 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T06:38:14.384 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.385 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.386 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.387 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.388 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.389 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.390 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.391 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T06:38:14.392 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.393 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.394 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.396 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T06:38:14.397 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.398 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.399 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.401 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.402 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.403 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-02-26T06:38:14.404 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T06:38:14.405 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.406 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.407 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.409 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.410 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.411 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.412 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T06:38:14.413 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.414 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.415 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.416 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.417 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-02-26T06:38:14.418 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.419 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.420 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.421 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.422 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.423 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.424 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.425 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.426 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.427 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.428 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.429 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.430 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.431 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.433 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.434 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.435 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.436 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:14.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.438 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.439 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.440 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.441 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-02-26T06:38:14.442 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.443 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-02-26T06:38:14.444 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.445 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-02-26T06:38:14.446 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-02-26T06:38:14.447 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.448 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-02-26T06:38:14.449 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.451 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.452 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.453 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.454 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.455 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.456 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.457 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.458 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.459 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.460 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.462 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.463 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-02-26T06:38:14.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T06:38:14.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.467 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.468 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.469 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.470 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.471 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.472 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.474 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.475 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.476 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.478 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.479 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.480 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-02-26T06:38:14.481 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.482 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-02-26T06:38:14.483 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.484 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-02-26T06:38:14.485 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.486 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-02-26T06:38:14.487 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.488 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.489 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.490 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.491 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.492 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.492 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.492 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.492 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.492 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.493 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.494 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.496 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.497 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.498 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.499 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.500 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.501 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.502 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.503 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.504 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:14.505 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.506 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.508 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.509 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.510 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.511 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.512 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.513 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.514 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.515 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.516 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.517 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.518 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.520 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.521 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.522 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.523 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.524 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.527 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-02-26T06:38:14.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.529 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.530 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.531 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.533 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.534 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.535 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.537 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.538 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.539 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-02-26T06:38:14.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-02-26T06:38:14.541 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.542 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.543 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.544 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.545 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.546 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.547 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.548 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.549 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.551 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-02-26T06:38:14.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.555 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.556 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.557 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-02-26T06:38:14.558 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.559 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.560 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.562 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.564 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.567 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.569 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.570 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.571 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.572 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.573 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.574 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.575 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.576 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.577 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.578 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.579 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.580 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.581 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.582 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.583 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.584 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.585 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.586 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.587 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.588 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.589 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.590 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.591 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.592 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.593 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.594 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.595 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-02-26T06:38:14.596 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.598 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.599 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.600 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.601 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.602 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.603 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.604 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.605 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-02-26T06:38:14.606 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.607 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.608 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.609 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.610 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.611 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.612 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.613 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.614 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-02-26T06:38:14.615 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.616 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.617 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.620 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.621 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.623 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.624 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.625 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.626 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.627 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.631 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.632 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.635 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.636 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.637 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.638 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-02-26T06:38:14.639 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-02-26T06:38:14.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.642 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-02-26T06:38:14.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.644 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.645 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.646 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.648 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.649 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.652 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.653 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-02-26T06:38:14.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-02-26T06:38:14.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:14.661 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.663 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.664 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-02-26T06:38:14.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.668 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.670 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.673 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.674 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-02-26T06:38:14.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.679 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.680 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.681 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.682 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.683 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.685 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.686 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.687 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.688 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-02-26T06:38:14.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.697 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.699 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.703 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.705 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.706 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "drive_group_interval": { 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "drive_group_interval", 2024-02-26T06:38:14.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300.0", 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T06:38:14.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.710 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.714 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.716 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.719 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-02-26T06:38:14.719 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.720 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.721 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-02-26T06:38:14.721 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.721 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.721 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.726 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.727 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.728 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.729 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.730 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.731 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.732 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-02-26T06:38:14.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.734 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.735 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-02-26T06:38:14.736 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.737 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.738 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.739 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.740 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.741 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.742 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.743 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.744 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.745 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-02-26T06:38:14.746 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.747 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.748 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.749 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-02-26T06:38:14.750 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T06:38:14.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.752 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.754 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.757 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.758 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.759 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.760 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.761 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.762 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.763 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.764 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.767 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.770 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.772 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.778 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.780 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.782 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.784 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.786 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.787 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.788 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.789 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.790 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-02-26T06:38:14.791 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.792 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.793 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.794 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.795 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.796 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.797 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-02-26T06:38:14.798 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T06:38:14.799 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.800 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.802 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.804 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.805 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.807 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-02-26T06:38:14.811 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.812 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.813 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.814 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.815 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.816 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.817 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.818 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.820 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.822 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.823 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.824 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.825 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.826 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.827 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.828 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.829 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.831 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.833 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.834 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.835 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.836 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.837 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.842 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-02-26T06:38:14.845 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.846 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-02-26T06:38:14.847 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-02-26T06:38:14.849 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.852 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.853 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.854 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.855 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-02-26T06:38:14.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.857 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.858 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-02-26T06:38:14.859 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-02-26T06:38:14.860 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.861 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.863 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-02-26T06:38:14.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-02-26T06:38:14.866 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: "services": { 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard": "https://172.21.15.6:8443/" 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: "always_on_modules": { 2024-02-26T06:38:14.867 INFO:teuthology.orchestra.run.smithi006.stdout: "octopus": [ 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-02-26T06:38:14.868 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "pacific": [ 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-02-26T06:38:14.869 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "quincy": [ 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-02-26T06:38:14.870 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-02-26T06:38:14.871 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "reef": [ 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-02-26T06:38:14.872 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "squid": [ 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-02-26T06:38:14.873 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "last_failure_osd_epoch": 88, 2024-02-26T06:38:14.874 INFO:teuthology.orchestra.run.smithi006.stdout: "active_clients": [ 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2786569881 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.875 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "libcephsqlite", 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-02-26T06:38:14.876 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1546140286 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:14.877 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-02-26T06:38:14.878 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3051366788 2024-02-26T06:38:14.878 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.878 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.879 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1398269967 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.880 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2676025661 2024-02-26T06:38:14.881 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-02-26T06:38:14.882 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1788370041 2024-02-26T06:38:14.883 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.883 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.883 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:14.883 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-02-26T06:38:14.883 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:38:14.884 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:13 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1917028729' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T06:38:14.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:14 smithi167 ceph-mon[36229]: pgmap v314: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-02-26T06:38:14.885 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:13 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1917028729' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T06:38:14.885 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:14 smithi090 ceph-mon[35681]: pgmap v314: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4dfc0a2290 msgr2=0x7f4dfc0a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4dfc0a2290 0x7f4dfc0a4750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e080058c0 tx=0x7f4e08005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 msgr2=0x7f4e1819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1819e310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4e04008000 tx=0x7f4e0402c690 comp rx=0 tx=0).stop 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 shutdown_connections 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f4dfc0a2290 0x7f4dfc0a4750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e181a2690 0x7f4e181a2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:14.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1810c530 0x7f4e1819e310 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:14.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 --2- 172.21.15.6:0/1917028729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e18107f80 0x7f4e1819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:14.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 >> 172.21.15.6:0/1917028729 conn(0x7f4e18076d20 msgr2=0x7f4e180776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:14.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 shutdown_connections 2024-02-26T06:38:14.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:12.585+0000 7f4e17200640 1 -- 172.21.15.6:0/1917028729 wait complete. 2024-02-26T06:38:14.893 DEBUG:tasks.check_counter:Getting stats from a 2024-02-26T06:38:14.893 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-26T06:38:14.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:13 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1917028729' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T06:38:14.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:14 smithi006 ceph-mon[31790]: pgmap v314: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-02-26T06:38:15.373 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:38:16.411 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:16 smithi006 ceph-mon[31790]: pgmap v315: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 244 B/s wr, 15 op/s 2024-02-26T06:38:16.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:16 smithi167 ceph-mon[36229]: pgmap v315: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 244 B/s wr, 15 op/s 2024-02-26T06:38:16.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:16 smithi090 ceph-mon[35681]: pgmap v315: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 244 B/s wr, 15 op/s 2024-02-26T06:38:17.738 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:38:17.738 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:17.738 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 450, 2024-02-26T06:38:17.738 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 563, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 202982, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 425120, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 10, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 5, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.134742842, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.058131195, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.061623671, 2024-02-26T06:38:17.739 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.009024931, 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 562, 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.096527962, 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000171757 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 888, 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000436373, 2024-02-26T06:38:17.740 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000491 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 496, 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 530, 2024-02-26T06:38:17.741 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 2047138, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 303131, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 16, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 7, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.222692672, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.090236850, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.164014234, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.013116906, 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:17.742 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 529, 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.071167971, 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000134533 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 377, 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000180773, 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000479 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.743 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1983728, 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 135680 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 836, 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 593, 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 367111, 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 616392, 2024-02-26T06:38:17.744 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 12, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 8, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.159451926, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.075746637, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.107063026, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000856564, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 590, 2024-02-26T06:38:17.745 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.071768724, 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000121641 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1202, 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000396209, 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000329 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.746 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 448080 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 1, 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-MDSRank": { 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-02-26T06:38:17.747 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 71, 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.064282861, 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000905392 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-02-26T06:38:17.748 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 32, 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.001389826, 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000043432 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: "request": 124, 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: "reply": 39, 2024-02-26T06:38:17.749 INFO:teuthology.orchestra.run.smithi006.stdout: "reply_latency": { 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 39, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.067243589, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.001724194 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_reply": 0, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "forward": 85, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:17.750 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_commit": 11, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_split": 0, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_merge": 0, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes": 65, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_top": 16, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_bottom": 49, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pinned": 40, 2024-02-26T06:38:17.751 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_expired": 12, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_with_caps": 14, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": 14, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "subtrees": 6, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse": 235, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_hit": 113, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_forward": 85, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_discover": 33, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:17.752 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_lock": 0, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "load_cent": 0, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "q": 0, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "exported": 1, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "exported_inodes": 0, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "imported": 2, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "imported_inodes": 32, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:17.753 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_backtrace_fetch": 13, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rfiles": 24, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rbytes": 8913130, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rsnaps": 1, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_fetch": 13, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:17.754 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_file_inodes": 13, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_revoke": 1, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_grant": 14, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:17.755 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps": 40, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps_dirty": 26, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_cap_release": 2, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "process_request_cap_release": 26 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache": { 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update": 3, 2024-02-26T06:38:17.756 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update_receipt": 125, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_try_discover": 76, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_send_discover": 99, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_handle_discover": 4, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays": 0, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_created": 13, 2024-02-26T06:38:17.757 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_enqueued": 13, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_migrated": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_started": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_completed": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:17.758 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_exportdir": 1, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_flush": 0, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_inodestats": 0 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log": { 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "evlrg": 0, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "evadd": 79, 2024-02-26T06:38:17.759 INFO:teuthology.orchestra.run.smithi006.stdout: "evex": 0, 2024-02-26T06:38:17.760 INFO:teuthology.orchestra.run.smithi006.stdout: "evtrm": 0, 2024-02-26T06:38:17.760 INFO:teuthology.orchestra.run.smithi006.stdout: "ev": 79, 2024-02-26T06:38:17.760 INFO:teuthology.orchestra.run.smithi006.stdout: "evexg": 0, 2024-02-26T06:38:17.760 INFO:teuthology.orchestra.run.smithi006.stdout: "evexd": 0, 2024-02-26T06:38:17.760 INFO:teuthology.orchestra.run.smithi006.stdout: "segadd": 2, 2024-02-26T06:38:17.760 INFO:teuthology.orchestra.run.smithi006.stdout: "segex": 0, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "segtrm": 0, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "seg": 2, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "segmjr": 2, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "segexg": 0, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "segexd": 0, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "expos": 4194304, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "wrpos": 4380731, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "rdpos": 0, 2024-02-26T06:38:17.761 INFO:teuthology.orchestra.run.smithi006.stdout: "jlat": { 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 15, 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.034955896, 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.002330393 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "replayed": 0 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_mem": { 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 66, 2024-02-26T06:38:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: "ino+": 86, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "ino-": 20, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "dir": 57, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "dir+": 82, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "dir-": 25, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "dn": 65, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "dn+": 105, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "dn-": 40, 2024-02-26T06:38:17.763 INFO:teuthology.orchestra.run.smithi006.stdout: "cap": 14, 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "cap+": 29, 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "cap-": 15, 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "rss": 52236, 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "heap": 207228 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_server": { 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_client_request": 128, 2024-02-26T06:38:17.764 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_server_request": 2, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_request": 124, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_session": 9, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_peer_request": 6, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "req_create_latency": { 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.765 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getattr_latency": { 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "req_link_latency": { 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookup_latency": { 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 25, 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.026198296, 2024-02-26T06:38:17.767 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.001047931 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.768 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupino_latency": { 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupname_latency": { 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.769 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.770 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lssnap_latency": { 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.772 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mkdir_latency": { 2024-02-26T06:38:17.772 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.772 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mknod_latency": { 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mksnap_latency": { 2024-02-26T06:38:17.773 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: "req_open_latency": { 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1, 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000489145, 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000489145 2024-02-26T06:38:17.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "req_readdir_latency": { 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rename_latency": { 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.775 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:17.776 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmdir_latency": { 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.777 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.779 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setattr_latency": { 2024-02-26T06:38:17.779 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.779 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.779 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.779 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.779 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setlayout_latency": { 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setxattr_latency": { 2024-02-26T06:38:17.781 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: "req_symlink_latency": { 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "req_unlink_latency": { 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 13, 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.040556148, 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.003119703 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:17.783 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_sessions": { 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: "session_count": 1, 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: "session_add": 1, 2024-02-26T06:38:17.784 INFO:teuthology.orchestra.run.smithi006.stdout: "session_remove": 0, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_open": 1, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_stale": 0, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "total_load": 38, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "average_load": 38, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "avg_session_uptime": 555, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.785 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:17.786 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:17.787 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:17.789 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 310611, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 176, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-02-26T06:38:17.790 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 191, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:17.791 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 284091, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 1914, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:17.792 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "objecter": { 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_active": 0, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_laggy": 0, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send": 95, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send_bytes": 203566, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_resend": 27, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_reply": 95, 2024-02-26T06:38:17.793 INFO:teuthology.orchestra.run.smithi006.stdout: "op_latency": { 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 95, 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.415995379, 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.004378898 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "op_inflight": 0, 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "oplen_avg": { 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 95, 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 124 2024-02-26T06:38:17.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "op": 95, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "op_r": 14, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "op_w": 81, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "op_rmw": 0, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "op_pg": 0, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_stat": 0, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_create": 10, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_read": 1, 2024-02-26T06:38:17.795 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_write": 16, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writefull": 11, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writesame": 0, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_append": 0, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_zero": 0, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_truncate": 0, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_delete": 25, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_mapext": 0, 2024-02-26T06:38:17.796 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_getxattr": 13, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_setxattr": 20, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_call": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_watch": 0, 2024-02-26T06:38:17.797 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_notify": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_other": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_active": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_send": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_resend": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_ping": 0, 2024-02-26T06:38:17.798 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_active": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_send": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_resend": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_active": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_send": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_resend": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_active": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_send": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_resend": 0, 2024-02-26T06:38:17.799 INFO:teuthology.orchestra.run.smithi006.stdout: "command_active": 0, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "command_send": 0, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "command_resend": 0, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "map_epoch": 96, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "map_full": 0, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "map_inc": 11, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_sessions": 11, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_open": 11, 2024-02-26T06:38:17.800 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_close": 0, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_laggy": 0, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_wr": 23, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_rd": 0, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_del": 5 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "oft": { 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_objs": 1, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:17.801 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_updates": 0, 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_removes": 0 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "osd-slow-ops": { 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_ops_count": 0 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "purge_queue": { 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops_high_water": 6, 2024-02-26T06:38:17.802 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing": 0, 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_high_water": 6, 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed_ops": 13, 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed": 13, 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:17.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 1778, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 2439555, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 1778, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 1778, 2024-02-26T06:38:17.804 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 2439555, 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:17.805 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 95, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 204879, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 95, 2024-02-26T06:38:17.806 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 204879, 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:17.807 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 1024, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 95, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 95, 2024-02-26T06:38:17.808 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 95, 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 95, 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:17.809 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 13, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1313, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 13, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:17.810 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 1, 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1313, 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:17.811 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle-0x561028376640": { 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 79, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 186427, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 79, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:17.812 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 15, 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 186427, 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.813 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:17.814 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:38:18.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:18 smithi006 ceph-mon[31790]: pgmap v316: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-02-26T06:38:18.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:18 smithi167 ceph-mon[36229]: pgmap v316: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-02-26T06:38:18.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:18 smithi090 ceph-mon[35681]: pgmap v316: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-02-26T06:38:18.723 INFO:tasks.check_counter:Daemon mds.a mds.exported=1 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds.imported=2 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=3 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=125 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=1 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:18.724 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-02-26T06:38:18.725 DEBUG:tasks.check_counter:Getting stats from d 2024-02-26T06:38:18.725 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-26T06:38:19.179 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:38:20.417 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:20 smithi090 ceph-mon[35681]: pgmap v317: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-02-26T06:38:20.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:20 smithi006 ceph-mon[31790]: pgmap v317: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-02-26T06:38:20.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:20 smithi167 ceph-mon[36229]: pgmap v317: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-02-26T06:38:21.658 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:38:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 267, 2024-02-26T06:38:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 269, 2024-02-26T06:38:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 12654874, 2024-02-26T06:38:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 83101, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 7, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.094070484, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.016026223, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.057198501, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.013032752, 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 267, 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.024066892, 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000090138 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 983, 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 953, 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 18921136, 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 357232, 2024-02-26T06:38:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 4, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.330750790, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.087338612, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.202548394, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.061146867, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 953, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.088295369, 2024-02-26T06:38:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000092649 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1965712, 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 134240 2024-02-26T06:38:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 1374, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 1539, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 15899718, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 875588, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 8, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 6, 2024-02-26T06:38:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.400958246, 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.172991817, 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.115271119, 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.077658970, 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1536, 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.142675204, 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000092887 2024-02-26T06:38:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 448896 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-02-26T06:38:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 1, 2024-02-26T06:38:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-02-26T06:38:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-MDSRank": { 2024-02-26T06:38:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-02-26T06:38:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-02-26T06:38:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 3883, 2024-02-26T06:38:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.211277519, 2024-02-26T06:38:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000054410 2024-02-26T06:38:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "request": 0, 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "reply": 0, 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "reply_latency": { 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_reply": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "forward": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_commit": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_split": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_merge": 0, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes": 7153, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_top": 196, 2024-02-26T06:38:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_bottom": 6957, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pinned": 6873, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_expired": 0, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_with_caps": 0, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": 0, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "subtrees": 13, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse": 0, 2024-02-26T06:38:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_hit": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_forward": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_discover": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_lock": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "load_cent": 0, 2024-02-26T06:38:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: "q": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "exported": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "exported_inodes": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "imported": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "imported_inodes": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rfiles": 0, 2024-02-26T06:38:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rbytes": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rsnaps": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps": 0, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_cap_release": 0, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "process_request_cap_release": 0 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache": { 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update": 0, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update_receipt": 0, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_try_discover": 0, 2024-02-26T06:38:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_send_discover": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_handle_discover": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_created": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_enqueued": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_migrated": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_started": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_completed": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_exportdir": 0, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_flush": 0, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_inodestats": 0 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log": { 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "evlrg": 0, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "evadd": 0, 2024-02-26T06:38:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: "evex": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "evtrm": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "ev": 20111, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "evexg": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "evexd": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "segadd": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "segex": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "segtrm": 0, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "seg": 26, 2024-02-26T06:38:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: "segmjr": 3, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "segexg": 0, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "segexd": 0, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "expos": 4194304, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "wrpos": 49034572, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "rdpos": 49033998, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "jlat": { 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: "replayed": 20111 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_mem": { 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 3247, 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: "ino+": 6766, 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: "ino-": 3519, 2024-02-26T06:38:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: "dir": 225, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "dir+": 277, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "dir-": 52, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "dn": 7153, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "dn+": 7970, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "dn-": 817, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "cap": 0, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "cap+": 0, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "cap-": 0, 2024-02-26T06:38:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: "rss": 68716, 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "heap": 215420 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_server": { 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_client_request": 0, 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_server_request": 0, 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_request": 0, 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_session": 0, 2024-02-26T06:38:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_peer_request": 0, 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "req_create_latency": { 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getattr_latency": { 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "req_link_latency": { 2024-02-26T06:38:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookup_latency": { 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupino_latency": { 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupname_latency": { 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lssnap_latency": { 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mkdir_latency": { 2024-02-26T06:38:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mknod_latency": { 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mksnap_latency": { 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: "req_open_latency": { 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: "req_readdir_latency": { 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rename_latency": { 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmdir_latency": { 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setattr_latency": { 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setlayout_latency": { 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setxattr_latency": { 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "req_symlink_latency": { 2024-02-26T06:38:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: "req_unlink_latency": { 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_sessions": { 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "session_count": 1, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "session_add": 4, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "session_remove": 3, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_open": 2, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_stale": 0, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "total_load": 0, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "average_load": 0, 2024-02-26T06:38:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: "avg_session_uptime": 3780, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 344965, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 3352, 2024-02-26T06:38:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 191, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 12605674, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 109185, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "objecter": { 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "op_active": 0, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "op_laggy": 0, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send": 2356, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send_bytes": 21, 2024-02-26T06:38:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: "op_resend": 3, 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "op_reply": 2356, 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "op_latency": { 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 2356, 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 2.615971031, 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.001110344 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "op_inflight": 0, 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "oplen_avg": { 2024-02-26T06:38:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 2356, 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 2357 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: "op": 2356, 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: "op_r": 2356, 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: "op_w": 0, 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: "op_rmw": 0, 2024-02-26T06:38:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: "op_pg": 0, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_stat": 1501, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_create": 0, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_read": 853, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_write": 0, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writefull": 0, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writesame": 0, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_append": 0, 2024-02-26T06:38:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_zero": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_truncate": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_delete": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_mapext": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_getxattr": 1, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_setxattr": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_call": 0, 2024-02-26T06:38:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_watch": 0, 2024-02-26T06:38:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_notify": 0, 2024-02-26T06:38:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls": 0, 2024-02-26T06:38:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_other": 0, 2024-02-26T06:38:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_active": 0, 2024-02-26T06:38:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_send": 0, 2024-02-26T06:38:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_resend": 0, 2024-02-26T06:38:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_ping": 0, 2024-02-26T06:38:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_active": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_send": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_resend": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_active": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_send": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_resend": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_active": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_send": 0, 2024-02-26T06:38:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_resend": 0, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "command_active": 0, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "command_send": 0, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "command_resend": 0, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "map_epoch": 96, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "map_full": 0, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "map_inc": 11, 2024-02-26T06:38:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_sessions": 9, 2024-02-26T06:38:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_open": 9, 2024-02-26T06:38:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_close": 0, 2024-02-26T06:38:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_laggy": 0, 2024-02-26T06:38:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_wr": 0, 2024-02-26T06:38:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_rd": 2, 2024-02-26T06:38:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_del": 0 2024-02-26T06:38:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: "oft": { 2024-02-26T06:38:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_objs": 0, 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_updates": 0, 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_removes": 0 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: "osd-slow-ops": { 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_ops_count": 0 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: "purge_queue": { 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing": 0, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_high_water": 0, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed_ops": 0, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed": 0, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 2620, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 47223538, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 2620, 2024-02-26T06:38:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 2620, 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 47223538, 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 2356, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 44846182, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 2356, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 44846182, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 1024, 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-02-26T06:38:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 2356, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 2356, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 2356, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 2356, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-02-26T06:38:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 0, 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle-0x55b71a260a00": { 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-02-26T06:38:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 0, 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 0, 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:21.738 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:38:22.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:22 smithi006 ceph-mon[31790]: pgmap v318: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 98 B/s wr, 12 op/s 2024-02-26T06:38:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:22 smithi167 ceph-mon[36229]: pgmap v318: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 98 B/s wr, 12 op/s 2024-02-26T06:38:22.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:22 smithi090 ceph-mon[35681]: pgmap v318: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 98 B/s wr, 12 op/s 2024-02-26T06:38:22.712 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-02-26T06:38:22.712 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-02-26T06:38:22.712 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-02-26T06:38:22.712 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-02-26T06:38:22.712 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-02-26T06:38:22.712 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:22.713 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:22.713 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2024-02-26T06:38:22.713 DEBUG:tasks.check_counter:Getting stats from g 2024-02-26T06:38:22.713 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-26T06:38:23.203 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:38:24.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:24 smithi006 ceph-mon[31790]: pgmap v319: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-02-26T06:38:24.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:24 smithi167 ceph-mon[36229]: pgmap v319: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-02-26T06:38:24.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:24 smithi090 ceph-mon[35681]: pgmap v319: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-02-26T06:38:25.390 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:38:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2024-02-26T06:38:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2024-02-26T06:38:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.005404522, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000940172, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 2, 2024-02-26T06:38:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 164, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 320, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 183760, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.033158817, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.024659000, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.003473522, 2024-02-26T06:38:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:38:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 161, 2024-02-26T06:38:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.024452962, 2024-02-26T06:38:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000151881 2024-02-26T06:38:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 183760 2024-02-26T06:38:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 232, 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 217, 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1847280, 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 129600, 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-02-26T06:38:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.105277817, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.033685001, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.088854796, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.010260368, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 217, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.024071347, 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000110927 2024-02-26T06:38:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1847280, 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 129600 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-02-26T06:38:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 149745, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 31, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-02-26T06:38:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-02-26T06:38:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-02-26T06:38:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-02-26T06:38:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-02-26T06:38:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 230, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1812043, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 230, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 230, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1812043, 2024-02-26T06:38:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:25.410 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:38:26.366 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-26T06:38:26.366 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-26T06:38:26.366 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-02-26T06:38:26.366 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-02-26T06:38:26.367 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-02-26T06:38:26.367 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-02-26T06:38:26.367 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-02-26T06:38:26.367 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-02-26T06:38:26.367 DEBUG:tasks.check_counter:Getting stats from j 2024-02-26T06:38:26.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-26T06:38:26.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:26 smithi006 ceph-mon[31790]: pgmap v320: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:26.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:26 smithi167 ceph-mon[36229]: pgmap v320: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:26.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:26 smithi090 ceph-mon[35681]: pgmap v320: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:26.846 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T06:38:28.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:28 smithi006 ceph-mon[31790]: pgmap v321: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:28.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:28 smithi167 ceph-mon[36229]: pgmap v321: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:28.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:28 smithi090 ceph-mon[35681]: pgmap v321: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:28.972 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-02-26T06:38:28.972 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:28.973 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 238, 2024-02-26T06:38:28.973 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 223, 2024-02-26T06:38:28.973 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1848912, 2024-02-26T06:38:28.973 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 133344, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.111029886, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.034677448, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.088910215, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.010225502, 2024-02-26T06:38:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 223, 2024-02-26T06:38:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.023439643, 2024-02-26T06:38:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000105110 2024-02-26T06:38:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:28.975 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:28.976 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:28.976 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:28.976 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.976 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1848912, 2024-02-26T06:38:28.976 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 133344 2024-02-26T06:38:28.976 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.977 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:28.977 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 2, 2024-02-26T06:38:28.977 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 170, 2024-02-26T06:38:28.977 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 320, 2024-02-26T06:38:28.977 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 184816, 2024-02-26T06:38:28.977 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.036086037, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.025854839, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.004163566, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 167, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.025197817, 2024-02-26T06:38:28.978 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000150885 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.979 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 184816 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2024-02-26T06:38:28.980 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.000861812, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000710617, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:28.981 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:28.982 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:28.983 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:28.984 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:28.985 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:28.987 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:28.988 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:28.989 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148714, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 30, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-02-26T06:38:28.990 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:28.991 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:28.991 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:28.991 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:28.991 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-02-26T06:38:28.991 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-02-26T06:38:28.991 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-02-26T06:38:28.992 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-02-26T06:38:28.993 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 236, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1812823, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 236, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 236, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1812823, 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-02-26T06:38:28.994 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-02-26T06:38:28.995 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-02-26T06:38:28.995 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-02-26T06:38:28.995 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:28.995 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-02-26T06:38:28.995 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-02-26T06:38:29.805 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-26T06:38:29.805 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-26T06:38:29.805 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-02-26T06:38:29.805 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-02-26T06:38:29.806 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-26T06:38:29.806 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-26T06:38:29.806 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-26T06:38:29.806 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-02-26T06:38:29.806 DEBUG:tasks.check_counter:Getting stats from b 2024-02-26T06:38:29.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-26T06:38:30.058 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:38:30.417 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:30 smithi090 ceph-mon[35681]: pgmap v322: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:30.418 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:30 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:38:30.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:30 smithi006 ceph-mon[31790]: pgmap v322: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:30.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:30 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:38:30.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:30 smithi167 ceph-mon[36229]: pgmap v322: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:30.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:30 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T06:38:31.738 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: pgmap v323: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:31.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:31 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.740 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-02-26T06:38:31.740 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:31.740 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 1579, 2024-02-26T06:38:31.740 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1437, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 644396, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 3092298, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 16, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.165431392, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.054651794, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.054713127, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.044867942, 2024-02-26T06:38:31.741 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1437, 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.587078915, 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000408544 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1826, 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000317152, 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000173 2024-02-26T06:38:31.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 311232 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 3245, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 3739, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 3090283, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 2078212, 2024-02-26T06:38:31.743 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 32, 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 10, 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.366056603, 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.105773117, 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.173652378, 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.102099951, 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:31.744 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3735, 2024-02-26T06:38:31.745 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.891653257, 2024-02-26T06:38:31.745 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000238729 2024-02-26T06:38:31.745 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.745 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:31.745 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4608, 2024-02-26T06:38:31.745 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000755613, 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000163 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 2349744, 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 260208 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2316, 2024-02-26T06:38:31.746 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1586, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1105149, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 1687141, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 36, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.224924393, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.075809380, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.108289396, 2024-02-26T06:38:31.747 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.049191034, 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1584, 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.388044923, 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000244977 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3247, 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000540310, 2024-02-26T06:38:31.748 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000166 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 91440, 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 225952 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-02-26T06:38:31.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 594, 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.441315584, 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000742955 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.750 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 54, 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002052636, 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000038011 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.751 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 1097, 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 1060, 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1060, 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.338221668, 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001262473 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 37, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 4, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 1, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 91, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 2, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 51, 2024-02-26T06:38:31.753 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 14, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 25, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 12, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 27, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 874, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 3, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 3, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 6, 2024-02-26T06:38:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 1227, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 862, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 34, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 12, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 2, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-02-26T06:38:31.755 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 20, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 864, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 1, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 2, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 35, 2024-02-26T06:38:31.756 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 3452, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 26459797, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 6, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 35, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 23, 2024-02-26T06:38:31.757 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 3, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 20, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 12, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 24, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 591, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:31.758 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 59, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 20, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 10, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 92 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 16, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 64, 2024-02-26T06:38:31.759 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 37, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 58, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 55, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 19, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 17, 2024-02-26T06:38:31.760 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 2, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 3, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 20, 2024-02-26T06:38:31.761 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 2, 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 831, 2024-02-26T06:38:31.762 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 767, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 767, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 64, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 7, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 11, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 10, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 10, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 1, 2024-02-26T06:38:31.763 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 1, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 1, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 6148158, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 6294108, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 171, 2024-02-26T06:38:31.764 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.577047884, 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.003374549 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 54, 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 642, 2024-02-26T06:38:31.765 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 588, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 57, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 110, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 53, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 51, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 947, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 896, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 3, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 605, 2024-02-26T06:38:31.766 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 602, 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 61516, 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 215420 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 1166, 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 3, 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 1097, 2024-02-26T06:38:31.767 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 55, 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 8, 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 552, 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.943043986, 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001708413 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-02-26T06:38:31.768 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 101, 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.026962614, 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000266956 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.769 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2, 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.003088027, 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001544013 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 5, 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002693394, 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000538678 2024-02-26T06:38:31.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.771 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.774 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-02-26T06:38:31.774 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10, 2024-02-26T06:38:31.774 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002429153, 2024-02-26T06:38:31.774 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000242915 2024-02-26T06:38:31.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: pgmap v323: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:31.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:31 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.776 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-02-26T06:38:31.776 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 45, 2024-02-26T06:38:31.776 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.090012982, 2024-02-26T06:38:31.776 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002000288 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10, 2024-02-26T06:38:31.777 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.090935142, 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.009093514 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 288, 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.069362577, 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000240842 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-02-26T06:38:31.778 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.008539164, 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002134791 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 19, 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.044399284, 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002336804 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.779 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.780 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:31.781 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.020619165, 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.005154791 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.782 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.001130109, 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001130109 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.783 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-02-26T06:38:31.784 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 8, 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.021352252, 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002669031 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.785 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000640936, 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000640936 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10, 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.013012883, 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001301288 2024-02-26T06:38:31.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.787 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 6, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 4, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 2, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 29, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 14, 2024-02-26T06:38:31.788 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 608, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 360, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 360, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:31.789 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:31.790 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:31.791 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:31.792 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 332068, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 170, 2024-02-26T06:38:31.793 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 968, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 11, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:31.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 191, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 265930, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2469, 2024-02-26T06:38:31.795 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-02-26T06:38:31.796 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 1179, 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 2560626, 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 37, 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 1179, 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1179, 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 27.768539375, 2024-02-26T06:38:31.797 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.023552620 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1179, 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1808 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 1179, 2024-02-26T06:38:31.798 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 50, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 1129, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 50, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 229, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 7, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 175, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 63, 2024-02-26T06:38:31.799 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 2, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 529, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:31.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 43, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 458, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-02-26T06:38:31.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-02-26T06:38:31.802 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-02-26T06:38:31.803 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 96, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 12, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 12, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 12, 2024-02-26T06:38:31.804 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 224, 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 10, 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 18 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-02-26T06:38:31.805 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-02-26T06:38:31.806 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 6, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 6, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 17, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 17, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 7136, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 4132012, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 7136, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:31.808 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 7136, 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 4132012, 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.809 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 1179, 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 2562275, 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1179, 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2562275, 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:31.812 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 1179, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 1179, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1179, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1179, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.813 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 17, 2024-02-26T06:38:31.814 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 1717, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 17, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 4, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1717, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.815 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x56428d140640": { 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:31.816 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 831, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2099804, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 831, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 171, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2099804, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:31.817 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:31.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:31.818 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.818 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:31.818 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-02-26T06:38:31.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: pgmap v323: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:31.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:31.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:31 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:32.501 INFO:tasks.check_counter:Daemon mds.b mds.exported=20 2024-02-26T06:38:32.501 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-02-26T06:38:32.501 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=16 2024-02-26T06:38:32.501 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=64 2024-02-26T06:38:32.502 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-02-26T06:38:32.502 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-02-26T06:38:32.502 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-02-26T06:38:32.502 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=2 2024-02-26T06:38:32.502 DEBUG:tasks.check_counter:Getting stats from e 2024-02-26T06:38:32.502 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-26T06:38:33.395 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:38:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: pgmap v324: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.615 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi167 to 174.1M 2024-02-26T06:38:33.616 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:33 smithi090 ceph-mon[35681]: Unable to set osd_memory_target on smithi167 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-02-26T06:38:33.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:33.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: pgmap v324: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi167 to 174.1M 2024-02-26T06:38:33.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:33 smithi006 ceph-mon[31790]: Unable to set osd_memory_target on smithi167 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-02-26T06:38:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:33.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: pgmap v324: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:33.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi167 to 174.1M 2024-02-26T06:38:33.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:33 smithi167 ceph-mon[36229]: Unable to set osd_memory_target on smithi167 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-02-26T06:38:35.057 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:34 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:35.057 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:34 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:35.077 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-02-26T06:38:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 1059, 2024-02-26T06:38:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1029, 2024-02-26T06:38:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 2231968, 2024-02-26T06:38:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 381802, 2024-02-26T06:38:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 5, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.138060137, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.050807745, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.062484959, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.027618709, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1029, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.052580894, 2024-02-26T06:38:35.079 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000051099 2024-02-26T06:38:35.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.080 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:35.080 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.080 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.080 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1966528, 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 136112 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 5, 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 173, 2024-02-26T06:38:35.081 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1036, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 532620, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 7, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.011981200, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.007274748, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.002444976, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000069416, 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:35.082 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 169, 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.021045957, 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000124532 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 531936 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 1380, 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1382, 2024-02-26T06:38:35.084 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 539192, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 429187, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.090158459, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.021618966, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.025242571, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.037229855, 2024-02-26T06:38:35.085 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1380, 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.036678508, 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000026578 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.086 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3753, 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.088548454, 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000023594 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.088 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-02-26T06:38:35.089 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 0, 2024-02-26T06:38:35.090 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 0, 2024-02-26T06:38:35.090 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-02-26T06:38:35.090 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.090 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.090 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 0, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 0, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-02-26T06:38:35.091 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 42, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 2, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 40, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 39, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 0, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-02-26T06:38:35.092 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 3, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-02-26T06:38:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:35.094 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:35.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:35.096 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-02-26T06:38:35.097 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:35.098 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-02-26T06:38:35.099 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 0, 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-02-26T06:38:35.100 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 79, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-02-26T06:38:35.101 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4380731, 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 4380157, 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 79 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 29, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 53, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 24, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 15, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 27, 2024-02-26T06:38:35.103 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 12, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 42, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 67, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 25, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 0, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 48644, 2024-02-26T06:38:35.104 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 215420 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 0, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 0, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 0, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.105 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.106 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.109 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-02-26T06:38:35.111 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.112 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.113 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-02-26T06:38:35.113 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.115 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:35.116 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.117 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:35.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:35.123 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-02-26T06:38:35.124 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 1, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 0, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 0, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 2487, 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:35.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:35.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:35.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:35.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:35.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 182898, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 70, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-02-26T06:38:35.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 191, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:35.131 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 115850, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 794, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-02-26T06:38:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 2172, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 21, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 3, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 2172, 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-02-26T06:38:35.133 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2172, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.195980235, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000550635 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2172, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 2173 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 2172, 2024-02-26T06:38:35.134 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 2172, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 0, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 1366, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 0, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 804, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 0, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 0, 2024-02-26T06:38:35.135 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 1, 2024-02-26T06:38:35.136 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-02-26T06:38:35.137 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-02-26T06:38:35.138 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-02-26T06:38:35.139 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 96, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 11, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 4, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 4, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-02-26T06:38:35.140 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 0, 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 2, 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 0 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 0, 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-02-26T06:38:35.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-02-26T06:38:35.142 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 2440, 2024-02-26T06:38:35.143 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2536007, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 2440, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2440, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2536007, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.144 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:35.145 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:35.146 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:35.146 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:35.146 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 2172, 2024-02-26T06:38:35.146 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 186432, 2024-02-26T06:38:35.146 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2172, 2024-02-26T06:38:35.146 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 186432, 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.147 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:35.148 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:35.148 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-02-26T06:38:35.148 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:35.148 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:35.148 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:35.148 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 2172, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 2172, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2172, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2172, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.149 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:35.150 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-02-26T06:38:35.151 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x5618a846a960": { 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:35.152 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-02-26T06:38:35.153 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:35.154 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-02-26T06:38:35.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:34 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:35.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:34 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:35.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:34 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:35.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:34 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:35.935 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:35.936 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-02-26T06:38:35.937 DEBUG:tasks.check_counter:Getting stats from h 2024-02-26T06:38:35.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-26T06:38:36.181 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:38:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: pgmap v325: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:36.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:36.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:35 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: pgmap v325: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:36.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:35 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: pgmap v325: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:36.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:36.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:35 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:37.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi006 to 177.8M 2024-02-26T06:38:37.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: Unable to set osd_memory_target on smithi006 to 186497024: error parsing value: Value '186497024' is below minimum 939524096 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:38:37.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:38:37.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:36 smithi167 ceph-mon[36229]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:37.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi006 to 177.8M 2024-02-26T06:38:37.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: Unable to set osd_memory_target on smithi006 to 186497024: error parsing value: Value '186497024' is below minimum 939524096 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.259 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:38:37.260 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:38:37.260 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:36 smithi090 ceph-mon[35681]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi006 to 177.8M 2024-02-26T06:38:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: Unable to set osd_memory_target on smithi006 to 186497024: error parsing value: Value '186497024' is below minimum 939524096 2024-02-26T06:38:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T06:38:37.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:36 smithi006 ceph-mon[31790]: from='mgr.15330 ' entity='mgr.x' 2024-02-26T06:38:37.836 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-02-26T06:38:37.836 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 826, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 477, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 321101, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 205395, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 18, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.061858814, 2024-02-26T06:38:37.837 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.021149861, 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.049939479, 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000696545, 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 475, 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.054895904, 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000115570 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:37.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1220, 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000203031, 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000166 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 353, 2024-02-26T06:38:37.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 432, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 2003884, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 470368, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 8, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.064540045, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.022115982, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.049673615, 2024-02-26T06:38:37.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.005893966, 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 432, 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.061828791, 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000143122 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 74, 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000009914, 2024-02-26T06:38:37.841 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000133 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1982256, 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 455184 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 408, 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 605, 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 186473, 2024-02-26T06:38:37.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 628972, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 13, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.064928135, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.031045429, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.027392178, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.003984769, 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:37.843 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 601, 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.038281291, 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000063695 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 931, 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000203067, 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000218 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.844 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 4736, 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 232848 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-02-26T06:38:37.845 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 65, 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.001627337, 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000025035 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:37.846 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2, 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000009531, 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000004765 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 87, 2024-02-26T06:38:37.847 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 6, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 6, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002598371, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000433061 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 81, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:37.848 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 11, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 43, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 13, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 30, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:37.849 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 22, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 12, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 1, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 1, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 4, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 167, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 62, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 76, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 28, 2024-02-26T06:38:37.850 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-02-26T06:38:37.851 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 24, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 8913130, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 1, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:37.852 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:38:37.853 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-02-26T06:38:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 124, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 76, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 94, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-02-26T06:38:37.855 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-02-26T06:38:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-02-26T06:38:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 8, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 8, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 2, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-02-26T06:38:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4196657, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-02-26T06:38:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 7, 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.033787597, 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.004826799 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 47, 2024-02-26T06:38:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 54, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 7, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 51, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 74, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 23, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 43, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 55, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 12, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 1, 2024-02-26T06:38:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 4, 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 3, 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 49952, 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 207228 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 90, 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 2, 2024-02-26T06:38:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 87, 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 24, 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 5, 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-02-26T06:38:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002170549, 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000542637 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2, 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000427822, 2024-02-26T06:38:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000213911 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.869 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.870 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.870 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-02-26T06:38:37.870 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.872 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.873 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.876 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.878 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 4, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 2, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 2, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 4, 2024-02-26T06:38:37.882 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 1, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 614, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:37.883 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:37.884 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:37.885 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 260491, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 139, 2024-02-26T06:38:37.887 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:37.888 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 191, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 210569, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 1212, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-02-26T06:38:37.889 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 47, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 13060, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 27, 2024-02-26T06:38:37.890 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 47, 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 47, 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.440992677, 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.009382822 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 47, 2024-02-26T06:38:37.891 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 71 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 47, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 0, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 47, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 0, 2024-02-26T06:38:37.892 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 10, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 0, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 7, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 9, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 6, 2024-02-26T06:38:37.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 20, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:37.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-02-26T06:38:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-02-26T06:38:37.896 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 96, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 11, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 8, 2024-02-26T06:38:37.897 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 8, 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 17, 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 0, 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 2 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-02-26T06:38:37.898 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-02-26T06:38:37.899 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 1583, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2350032, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 1583, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:37.901 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1583, 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2350032, 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:37.902 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 47, 2024-02-26T06:38:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 13060, 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 47, 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 13060, 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 47, 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 47, 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 47, 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 47, 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:37.907 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x5618b1986640": { 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 8, 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2353, 2024-02-26T06:38:37.909 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 8, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 7, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2353, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:37.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:37.911 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.911 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:37.911 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-02-26T06:38:38.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:37 smithi090 ceph-mon[35681]: Adjusting osd_memory_target on smithi090 to 174.1M 2024-02-26T06:38:38.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:37 smithi090 ceph-mon[35681]: Unable to set osd_memory_target on smithi090 to 182612684: error parsing value: Value '182612684' is below minimum 939524096 2024-02-26T06:38:38.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:37 smithi090 ceph-mon[35681]: pgmap v326: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:37 smithi167 ceph-mon[36229]: Adjusting osd_memory_target on smithi090 to 174.1M 2024-02-26T06:38:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:37 smithi167 ceph-mon[36229]: Unable to set osd_memory_target on smithi090 to 182612684: error parsing value: Value '182612684' is below minimum 939524096 2024-02-26T06:38:38.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:37 smithi167 ceph-mon[36229]: pgmap v326: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:38.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:37 smithi006 ceph-mon[31790]: Adjusting osd_memory_target on smithi090 to 174.1M 2024-02-26T06:38:38.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:37 smithi006 ceph-mon[31790]: Unable to set osd_memory_target on smithi090 to 182612684: error parsing value: Value '182612684' is below minimum 939524096 2024-02-26T06:38:38.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:37 smithi006 ceph-mon[31790]: pgmap v326: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:38.706 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-02-26T06:38:38.706 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-02-26T06:38:38.706 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-02-26T06:38:38.707 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=124 2024-02-26T06:38:38.707 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=1 2024-02-26T06:38:38.707 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:38.707 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:38.707 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-02-26T06:38:38.707 DEBUG:tasks.check_counter:Getting stats from k 2024-02-26T06:38:38.708 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-26T06:38:38.943 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T06:38:40.235 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:39 smithi090 ceph-mon[35681]: pgmap v327: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:40.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:39 smithi167 ceph-mon[36229]: pgmap v327: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:40.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:39 smithi006 ceph-mon[31790]: pgmap v327: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:40.635 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-02-26T06:38:40.635 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:40.635 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 1601, 2024-02-26T06:38:40.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1601, 2024-02-26T06:38:40.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 482592, 2024-02-26T06:38:40.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 497116, 2024-02-26T06:38:40.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 6, 2024-02-26T06:38:40.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-02-26T06:38:40.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.146315786, 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.054557021, 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.035019675, 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.042651948, 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1600, 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.064044318, 2024-02-26T06:38:40.637 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000040027 2024-02-26T06:38:40.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.638 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:40.638 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.638 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.638 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.639 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-26T06:38:40.639 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T06:38:40.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.639 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:40.640 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 660, 2024-02-26T06:38:40.640 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 830, 2024-02-26T06:38:40.640 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 162840, 2024-02-26T06:38:40.640 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 659282, 2024-02-26T06:38:40.640 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 7, 2024-02-26T06:38:40.640 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.070217262, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.021999642, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.016465312, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.024754369, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 827, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.050390733, 2024-02-26T06:38:40.641 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000060931 2024-02-26T06:38:40.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.642 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:40.642 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.642 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.642 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.643 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:38:40.643 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 454576 2024-02-26T06:38:40.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.643 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:40.643 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 270, 2024-02-26T06:38:40.643 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 242, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1964812, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 138471, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 3, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.052555759, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.015214958, 2024-02-26T06:38:40.644 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.043572750, 2024-02-26T06:38:40.645 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.005957385, 2024-02-26T06:38:40.645 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:40.645 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 241, 2024-02-26T06:38:40.645 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.014056953, 2024-02-26T06:38:40.645 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000058327 2024-02-26T06:38:40.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1964544, 2024-02-26T06:38:40.646 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 138160 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:40.647 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3708, 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.098430826, 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000026545 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-02-26T06:38:40.648 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 0, 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 0, 2024-02-26T06:38:40.649 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 0, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:40.650 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 0, 2024-02-26T06:38:40.651 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 1, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:40.652 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-02-26T06:38:40.653 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:40.654 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:40.655 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-02-26T06:38:40.656 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-02-26T06:38:40.657 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-02-26T06:38:40.658 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-02-26T06:38:40.659 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 8, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-02-26T06:38:40.660 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4196657, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 4196516, 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-02-26T06:38:40.661 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 8 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 3, 2024-02-26T06:38:40.662 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 3, 2024-02-26T06:38:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 0, 2024-02-26T06:38:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 1, 2024-02-26T06:38:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 1, 2024-02-26T06:38:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 0, 2024-02-26T06:38:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 0, 2024-02-26T06:38:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 0, 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 0, 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 0, 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 48052, 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 215420 2024-02-26T06:38:40.664 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 0, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 0, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 0, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.665 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.666 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.666 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-02-26T06:38:40.666 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.666 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.667 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-02-26T06:38:40.668 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.668 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.668 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.668 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.668 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-02-26T06:38:40.668 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.669 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.670 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-02-26T06:38:40.670 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.670 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.670 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.671 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-02-26T06:38:40.671 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.671 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.671 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.672 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.673 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.673 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-02-26T06:38:40.673 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.673 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.673 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.674 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-02-26T06:38:40.674 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.674 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.674 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.675 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-02-26T06:38:40.675 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-02-26T06:38:40.677 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.678 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:40.679 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.680 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.681 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-02-26T06:38:40.682 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.683 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-02-26T06:38:40.684 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.685 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.686 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-02-26T06:38:40.686 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.686 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.686 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:40.687 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 4, 2024-02-26T06:38:40.688 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 2, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 3, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 0, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 6184, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-02-26T06:38:40.689 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:40.690 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:40.691 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:40.691 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:40.691 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:40.691 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:40.691 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:40.692 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:40.693 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 190139, 2024-02-26T06:38:40.694 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 52, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:40.695 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 191, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 8320, 2024-02-26T06:38:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 13, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-02-26T06:38:40.697 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 2256, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 21, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 3, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 2256, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2256, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.176980563, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000521711 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.698 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2256, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 2257 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 2256, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 2256, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 0, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-02-26T06:38:40.699 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 1450, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 804, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-02-26T06:38:40.700 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 1, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:40.701 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:40.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-02-26T06:38:40.703 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-02-26T06:38:40.703 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-02-26T06:38:40.703 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-02-26T06:38:40.703 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-02-26T06:38:40.703 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-02-26T06:38:40.703 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-02-26T06:38:40.704 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 96, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 11, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 5, 2024-02-26T06:38:40.705 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 5, 2024-02-26T06:38:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-02-26T06:38:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-02-26T06:38:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 0, 2024-02-26T06:38:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 2, 2024-02-26T06:38:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 0 2024-02-26T06:38:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.707 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-02-26T06:38:40.707 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 0, 2024-02-26T06:38:40.707 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:40.707 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-02-26T06:38:40.707 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-02-26T06:38:40.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.708 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-02-26T06:38:40.708 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-02-26T06:38:40.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.708 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-02-26T06:38:40.708 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:40.708 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:40.709 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 2527, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2366060, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 2527, 2024-02-26T06:38:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:40.711 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:40.711 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2527, 2024-02-26T06:38:40.711 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2366060, 2024-02-26T06:38:40.711 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:40.711 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.711 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-02-26T06:38:40.712 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:40.713 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:40.713 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:40.713 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:40.713 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 2256, 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 2358, 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2256, 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2358, 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.714 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-02-26T06:38:40.715 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 2256, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 2256, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2256, 2024-02-26T06:38:40.716 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2256, 2024-02-26T06:38:40.717 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:40.717 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.717 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.717 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.717 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:40.718 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.719 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.720 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.720 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-02-26T06:38:40.720 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x55c7a11f4960": { 2024-02-26T06:38:40.720 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-02-26T06:38:40.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-02-26T06:38:40.721 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-02-26T06:38:40.721 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-02-26T06:38:40.721 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-02-26T06:38:40.721 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:40.721 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-02-26T06:38:40.722 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.723 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-02-26T06:38:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-02-26T06:38:41.494 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:41.495 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-02-26T06:38:41.495 DEBUG:tasks.check_counter:Getting stats from c 2024-02-26T06:38:41.496 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-26T06:38:41.750 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:38:42.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:41 smithi167 ceph-mon[36229]: pgmap v328: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:42.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:41 smithi090 ceph-mon[35681]: pgmap v328: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:42.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:41 smithi006 ceph-mon[31790]: pgmap v328: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:43.716 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-02-26T06:38:43.716 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:43.716 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1488, 2024-02-26T06:38:43.716 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1271, 2024-02-26T06:38:43.716 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1391316, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 2745983, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 16, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.127446770, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.054870206, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.066323075, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.011642099, 2024-02-26T06:38:43.717 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1270, 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.056198499, 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000044250 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2275, 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000457538, 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000201 2024-02-26T06:38:43.718 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 507, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 632, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2036067, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 586651, 2024-02-26T06:38:43.719 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 13, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.072417914, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.024050316, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.049100533, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.009005903, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 631, 2024-02-26T06:38:43.720 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.069940114, 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000110840 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 89, 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000013619, 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000153 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1971488, 2024-02-26T06:38:43.721 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 454944 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2978, 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 4938, 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1023953, 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 3515183, 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 12, 2024-02-26T06:38:43.722 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.170672421, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.080561976, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.124103639, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.004996873, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4934, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.107531600, 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000021794 2024-02-26T06:38:43.723 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5905, 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000889727, 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000150 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 221968 2024-02-26T06:38:43.724 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 154, 2024-02-26T06:38:43.725 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.617774263, 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004011521 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 866, 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.048358902, 2024-02-26T06:38:43.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000055841 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 1541, 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 1540, 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1540, 2024-02-26T06:38:43.727 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.293890360, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000840188 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 1, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2024-02-26T06:38:43.728 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 810, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 22, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 788, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 776, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 12, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 72, 2024-02-26T06:38:43.729 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 72, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 16, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 2499, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 2424, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 1, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 41, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-02-26T06:38:43.730 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 1, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 18, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 773, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:43.731 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 24, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 8913130, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 1, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:43.732 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 4, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 683, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:43.733 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 1195, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 968, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 11, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 896 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 54, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 80, 2024-02-26T06:38:43.734 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 51, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 67, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 55, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 447, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 447, 2024-02-26T06:38:43.735 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 1, 2024-02-26T06:38:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 2636, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-02-26T06:38:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 2636, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 5, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 5, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-02-26T06:38:43.738 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 9046688, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 51, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.494355169, 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.009693238 2024-02-26T06:38:43.739 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 581, 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 1035, 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 454, 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 73, 2024-02-26T06:38:43.740 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 78, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 5, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 810, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 1282, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 472, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 72, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 984, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 912, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 63740, 2024-02-26T06:38:43.741 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207228 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 1559, 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 2, 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 1541, 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 13, 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 11, 2024-02-26T06:38:43.742 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 220, 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.135653082, 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000616604 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.744 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 869, 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.359030391, 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000413153 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000134158, 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000134158 2024-02-26T06:38:43.746 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.748 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.749 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.749 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.749 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3, 2024-02-26T06:38:43.751 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000668099, 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000222699 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.752 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.753 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.754 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:43.755 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.756 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.757 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.758 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-02-26T06:38:43.759 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 447, 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.798404630, 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001786140 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.760 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-02-26T06:38:43.761 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 1479, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 739, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 629, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:43.762 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:43.763 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:43.765 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:43.766 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 337493, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 698, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 792, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 9, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:43.767 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 191, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-02-26T06:38:43.768 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 2063929, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 25528, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-02-26T06:38:43.769 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 588, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 4996517, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 28, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 588, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 588, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.976963521, 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003362182 2024-02-26T06:38:43.770 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 588, 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 630 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 588, 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 10, 2024-02-26T06:38:43.771 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 578, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 10, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 61, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 16, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-02-26T06:38:43.772 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 460, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-02-26T06:38:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-02-26T06:38:43.774 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-02-26T06:38:43.775 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-02-26T06:38:43.776 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 96, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 11, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-02-26T06:38:43.777 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 47, 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 6 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-02-26T06:38:43.778 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 6, 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-02-26T06:38:43.779 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 6, 2024-02-26T06:38:43.780 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 447, 2024-02-26T06:38:43.780 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 447, 2024-02-26T06:38:43.780 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:43.780 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.781 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:43.781 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:43.781 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:43.781 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 4969, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 3986113, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 4969, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4969, 2024-02-26T06:38:43.782 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 3986113, 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:43.783 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 588, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 5041664, 2024-02-26T06:38:43.784 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 588, 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 5041664, 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.785 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:43.786 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 588, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 588, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 588, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 588, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 447, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 45147, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 447, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 10, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 45147, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x564a2c410640": { 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2636, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 4852384, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2636, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 51, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4852384, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:43.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:43.792 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.792 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:43.792 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-02-26T06:38:44.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:43 smithi090 ceph-mon[35681]: pgmap v329: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:44.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:43 smithi167 ceph-mon[36229]: pgmap v329: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:44.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:43 smithi006 ceph-mon[31790]: pgmap v329: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:44.547 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds.imported=18 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=54 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=80 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=1 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:44.548 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-02-26T06:38:44.548 DEBUG:tasks.check_counter:Getting stats from f 2024-02-26T06:38:44.549 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-26T06:38:44.789 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:38:46.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:45 smithi167 ceph-mon[36229]: pgmap v330: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:46.364 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:45 smithi090 ceph-mon[35681]: pgmap v330: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:46.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:45 smithi006 ceph-mon[31790]: pgmap v330: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1677, 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1679, 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2597873, 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 521557, 2024-02-26T06:38:46.448 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.167404196, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.053488521, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.044606590, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.053103214, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1677, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.070156717, 2024-02-26T06:38:46.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000041834 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:46.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 272, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 244, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1965816, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 138382, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.051509385, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.014981526, 2024-02-26T06:38:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.042064045, 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.006001565, 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 243, 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.018694312, 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000076931 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:46.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1964160, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 137072 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 796, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 963, 2024-02-26T06:38:46.453 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 197803, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 703739, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.088431385, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.034973932, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.018039689, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.025957680, 2024-02-26T06:38:46.454 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 961, 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.069288179, 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000072100 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 458048 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-02-26T06:38:46.456 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4066, 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.124596888, 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000030643 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:46.457 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-02-26T06:38:46.458 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-02-26T06:38:46.459 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 41, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 18, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 23, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 13, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-02-26T06:38:46.460 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 4, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-02-26T06:38:46.461 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:46.462 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 3452, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 26459797, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 6, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:46.463 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:46.464 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-02-26T06:38:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:46.466 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-02-26T06:38:46.467 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-02-26T06:38:46.468 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 831, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 11, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-02-26T06:38:46.469 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 6148158, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 6294108, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 6293996, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.470 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 831 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 22, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 609, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 587, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 16, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 70, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 54, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 41, 2024-02-26T06:38:46.471 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 623, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 582, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 50556, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215420 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.472 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.474 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-02-26T06:38:46.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.477 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.478 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.479 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.480 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.480 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.480 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.481 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.482 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.483 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:46.485 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.486 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.487 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 2, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 6, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 4, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 4, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-02-26T06:38:46.491 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 8211, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:46.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:46.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:46.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:46.495 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 190637, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 66, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-02-26T06:38:46.496 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 191, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-02-26T06:38:46.497 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 113063, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 672, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-02-26T06:38:46.498 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 2471, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 2471, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2471, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.123979248, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000454868 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.499 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-02-26T06:38:46.500 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-02-26T06:38:46.500 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2471, 2024-02-26T06:38:46.500 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2472 2024-02-26T06:38:46.500 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.500 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 2471, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 2471, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1590, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 878, 2024-02-26T06:38:46.501 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-02-26T06:38:46.502 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:46.503 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:46.503 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2024-02-26T06:38:46.503 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-02-26T06:38:46.503 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-02-26T06:38:46.504 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:46.505 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-02-26T06:38:46.505 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-02-26T06:38:46.505 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-02-26T06:38:46.505 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-02-26T06:38:46.505 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-02-26T06:38:46.506 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-02-26T06:38:46.507 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-02-26T06:38:46.507 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-02-26T06:38:46.507 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-02-26T06:38:46.507 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-02-26T06:38:46.507 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-02-26T06:38:46.507 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 96, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 11, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 5, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 5, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-02-26T06:38:46.508 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-02-26T06:38:46.509 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-02-26T06:38:46.509 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-02-26T06:38:46.509 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.509 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-02-26T06:38:46.509 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-02-26T06:38:46.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.511 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-02-26T06:38:46.511 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:46.511 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:38:46.511 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-02-26T06:38:46.511 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-02-26T06:38:46.511 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-02-26T06:38:46.512 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-02-26T06:38:46.512 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:46.512 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.512 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:46.512 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:46.513 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:46.513 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:46.513 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2741, 2024-02-26T06:38:46.513 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 4498106, 2024-02-26T06:38:46.513 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:46.514 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2741, 2024-02-26T06:38:46.514 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:46.514 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:46.514 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2741, 2024-02-26T06:38:46.515 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4498106, 2024-02-26T06:38:46.515 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:46.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.515 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:46.516 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:46.517 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:46.517 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:46.517 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:46.517 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:46.518 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2471, 2024-02-26T06:38:46.518 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2099814, 2024-02-26T06:38:46.518 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2471, 2024-02-26T06:38:46.518 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2099814, 2024-02-26T06:38:46.518 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:46.519 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-02-26T06:38:46.520 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:46.520 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:46.520 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:46.520 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:46.520 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:46.520 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2471, 2024-02-26T06:38:46.521 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2471, 2024-02-26T06:38:46.521 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2471, 2024-02-26T06:38:46.521 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2471, 2024-02-26T06:38:46.521 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:46.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.521 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:46.522 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:46.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x559dab744960": { 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:46.524 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-02-26T06:38:46.525 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:46.526 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:46.526 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:46.526 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:46.526 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.526 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:46.526 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-02-26T06:38:47.257 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-26T06:38:47.257 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-26T06:38:47.258 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-02-26T06:38:47.258 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-02-26T06:38:47.258 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-02-26T06:38:47.258 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:47.258 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:47.258 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-02-26T06:38:47.258 DEBUG:tasks.check_counter:Getting stats from i 2024-02-26T06:38:47.259 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-26T06:38:47.501 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:38:48.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:47 smithi167 ceph-mon[36229]: pgmap v331: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:48.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:47 smithi090 ceph-mon[35681]: pgmap v331: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:48.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:47 smithi006 ceph-mon[31790]: pgmap v331: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:49.257 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-02-26T06:38:49.257 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:49.258 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1603, 2024-02-26T06:38:49.258 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2476, 2024-02-26T06:38:49.258 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 453352, 2024-02-26T06:38:49.258 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 14517147, 2024-02-26T06:38:49.259 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 23, 2024-02-26T06:38:49.259 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-02-26T06:38:49.261 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.161007055, 2024-02-26T06:38:49.261 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.082107146, 2024-02-26T06:38:49.261 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.047659088, 2024-02-26T06:38:49.261 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.026269211, 2024-02-26T06:38:49.262 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:49.262 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2475, 2024-02-26T06:38:49.262 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.129282334, 2024-02-26T06:38:49.262 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000052235 2024-02-26T06:38:49.263 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.263 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:49.263 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2091, 2024-02-26T06:38:49.264 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000331545, 2024-02-26T06:38:49.264 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000158 2024-02-26T06:38:49.265 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.266 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 4928, 2024-02-26T06:38:49.266 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 317984 2024-02-26T06:38:49.267 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.267 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:49.267 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2513, 2024-02-26T06:38:49.267 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2856, 2024-02-26T06:38:49.268 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2630776, 2024-02-26T06:38:49.268 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 24392245, 2024-02-26T06:38:49.268 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-02-26T06:38:49.268 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-02-26T06:38:49.269 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.272996004, 2024-02-26T06:38:49.269 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.130218427, 2024-02-26T06:38:49.269 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.096398299, 2024-02-26T06:38:49.269 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.053643330, 2024-02-26T06:38:49.269 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:49.269 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2855, 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.113571802, 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000039779 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1313, 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000200829, 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000152 2024-02-26T06:38:49.270 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1983392, 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 207552 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 19708, 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 37763, 2024-02-26T06:38:49.271 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 7625109, 2024-02-26T06:38:49.272 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 20477368, 2024-02-26T06:38:49.272 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-02-26T06:38:49.272 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-02-26T06:38:49.272 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 1.239517724, 2024-02-26T06:38:49.273 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.578076873, 2024-02-26T06:38:49.273 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 1.201776693, 2024-02-26T06:38:49.273 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.037171924, 2024-02-26T06:38:49.273 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:49.273 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 37760, 2024-02-26T06:38:49.273 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.972765626, 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000025761 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 42996, 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.006617810, 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000153 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.274 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2304, 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 145952 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:49.275 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1239, 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 8.138300379, 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006568442 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:49.277 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5093, 2024-02-26T06:38:49.277 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.038732307, 2024-02-26T06:38:49.277 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000203952 2024-02-26T06:38:49.277 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 9349, 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 9294, 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 9294, 2024-02-26T06:38:49.278 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 23.237790316, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002500300 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 55, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 184, 2024-02-26T06:38:49.279 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 21, 2024-02-26T06:38:49.280 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 1, 2024-02-26T06:38:49.280 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 4850, 2024-02-26T06:38:49.280 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 494, 2024-02-26T06:38:49.280 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 4330, 2024-02-26T06:38:49.280 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 26, 2024-02-26T06:38:49.280 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 4144, 2024-02-26T06:38:49.281 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 826, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 2962, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 2962, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 15, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 9864, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 9481, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 55, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 23, 2024-02-26T06:38:49.282 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 21, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 807, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 22, 2024-02-26T06:38:49.283 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 864, 2024-02-26T06:38:49.284 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:49.284 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 558, 2024-02-26T06:38:49.285 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:49.286 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 24, 2024-02-26T06:38:49.286 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 8913130, 2024-02-26T06:38:49.286 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 1, 2024-02-26T06:38:49.287 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 558, 2024-02-26T06:38:49.287 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:49.287 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:49.287 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:49.288 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 38, 2024-02-26T06:38:49.288 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:49.288 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 38, 2024-02-26T06:38:49.289 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 520, 2024-02-26T06:38:49.289 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 6, 2024-02-26T06:38:49.289 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 67, 2024-02-26T06:38:49.289 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 9573, 2024-02-26T06:38:49.290 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 8, 2024-02-26T06:38:49.290 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:49.290 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:49.291 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 7381, 2024-02-26T06:38:49.291 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 5798, 2024-02-26T06:38:49.291 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 822, 2024-02-26T06:38:49.292 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 2960 2024-02-26T06:38:49.292 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.292 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-02-26T06:38:49.293 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 84, 2024-02-26T06:38:49.293 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 12, 2024-02-26T06:38:49.293 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 8, 2024-02-26T06:38:49.294 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 28, 2024-02-26T06:38:49.294 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 232, 2024-02-26T06:38:49.294 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 1, 2024-02-26T06:38:49.294 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:49.294 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:49.295 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 2731, 2024-02-26T06:38:49.295 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 2730, 2024-02-26T06:38:49.295 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:49.296 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-02-26T06:38:49.296 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:49.297 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:49.297 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:49.298 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-02-26T06:38:49.298 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-02-26T06:38:49.298 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:49.298 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 21, 2024-02-26T06:38:49.298 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-02-26T06:38:49.299 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 22, 2024-02-26T06:38:49.299 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:49.299 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-02-26T06:38:49.299 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.299 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-02-26T06:38:49.299 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-02-26T06:38:49.300 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 20111, 2024-02-26T06:38:49.300 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-02-26T06:38:49.300 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-02-26T06:38:49.300 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 20111, 2024-02-26T06:38:49.300 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-02-26T06:38:49.300 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-02-26T06:38:49.301 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 26, 2024-02-26T06:38:49.301 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-02-26T06:38:49.301 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-02-26T06:38:49.301 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 26, 2024-02-26T06:38:49.301 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 3, 2024-02-26T06:38:49.301 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-02-26T06:38:49.302 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-02-26T06:38:49.302 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-02-26T06:38:49.302 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 49034572, 2024-02-26T06:38:49.302 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-02-26T06:38:49.302 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 378, 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 9.666877497, 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.025573749 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.303 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 3259, 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 6768, 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 3509, 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 235, 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 280, 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 45, 2024-02-26T06:38:49.304 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 4850, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 8142, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 3292, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 2962, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 6491, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 3529, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 93632, 2024-02-26T06:38:49.305 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207228 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 9647, 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 11, 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 9349, 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 41, 2024-02-26T06:38:49.306 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 38, 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5871, 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 14.875766024, 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002533770 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 11, 2024-02-26T06:38:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.451543333, 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.041049393 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.308 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 332, 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.094109416, 2024-02-26T06:38:49.309 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000283462 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-02-26T06:38:49.310 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000085277, 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000085277 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:49.311 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.312 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.313 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-02-26T06:38:49.313 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.313 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 40, 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.031239803, 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000780995 2024-02-26T06:38:49.314 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 149, 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.047712376, 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000320217 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-02-26T06:38:49.316 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 103, 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.625396147, 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006071807 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 62, 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.097363393, 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001570377 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.317 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 22, 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.156530890, 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.052569585 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.319 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.320 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.320 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.320 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:49.320 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.320 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.320 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8, 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.087816150, 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.010977018 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.321 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8, 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.003145172, 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000393146 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.322 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.323 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.324 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.324 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.324 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-02-26T06:38:49.324 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6, 2024-02-26T06:38:49.324 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.005639551, 2024-02-26T06:38:49.324 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000939925 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2681, 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 5.761442784, 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002148990 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:49.325 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.326 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 4, 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 3, 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 6127, 2024-02-26T06:38:49.327 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 6127, 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 635, 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:49.328 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:49.329 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:49.331 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:49.332 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:49.332 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:49.332 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:49.332 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:49.332 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:49.333 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:49.334 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:49.334 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 438094, 2024-02-26T06:38:49.334 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 3220, 2024-02-26T06:38:49.334 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 1584, 2024-02-26T06:38:49.334 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 18, 2024-02-26T06:38:49.334 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-02-26T06:38:49.335 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-02-26T06:38:49.335 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:49.335 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:49.335 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:49.335 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:49.336 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:49.336 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 191, 2024-02-26T06:38:49.336 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:49.336 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:49.336 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:49.336 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 12406193, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 142105, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-02-26T06:38:49.337 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 4101, 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 48115319, 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 4101, 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-02-26T06:38:49.338 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4101, 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 11.696783702, 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002852178 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4101, 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 4378 2024-02-26T06:38:49.339 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.340 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 4101, 2024-02-26T06:38:49.340 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 601, 2024-02-26T06:38:49.340 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 3500, 2024-02-26T06:38:49.340 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-02-26T06:38:49.341 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-02-26T06:38:49.341 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-02-26T06:38:49.341 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-02-26T06:38:49.341 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 43, 2024-02-26T06:38:49.342 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 421, 2024-02-26T06:38:49.342 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 34, 2024-02-26T06:38:49.342 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-02-26T06:38:49.342 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-02-26T06:38:49.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-02-26T06:38:49.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 5, 2024-02-26T06:38:49.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 2776, 2024-02-26T06:38:49.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-02-26T06:38:49.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:49.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:49.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 558, 2024-02-26T06:38:49.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-02-26T06:38:49.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:49.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 8, 2024-02-26T06:38:49.345 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-02-26T06:38:49.346 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 96, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-02-26T06:38:49.347 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 11, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 472, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 31 2024-02-26T06:38:49.348 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-02-26T06:38:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.350 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-02-26T06:38:49.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:49.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 6, 2024-02-26T06:38:49.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 6, 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 2730, 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 2730, 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:49.351 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 23820, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 8546492, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 23820, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:49.352 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 23820, 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 8546492, 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.353 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:49.354 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 4101, 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 48391737, 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4101, 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 48391737, 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.355 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:49.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 4101, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 4101, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4101, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4101, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.357 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.358 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.358 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.358 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.358 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:49.358 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:49.358 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2730, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 276418, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2730, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:49.359 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 43, 2024-02-26T06:38:49.360 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 276418, 2024-02-26T06:38:49.360 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:49.360 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.360 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.360 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.361 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.361 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:49.361 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x5599d3ed6640": { 2024-02-26T06:38:49.361 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:49.361 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:49.361 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 20111, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 44840268, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 20111, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 378, 2024-02-26T06:38:49.362 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 44840268, 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:49.363 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-02-26T06:38:49.989 INFO:tasks.check_counter:Daemon mds.i mds.exported=21 2024-02-26T06:38:49.989 INFO:tasks.check_counter:Daemon mds.i mds.imported=22 2024-02-26T06:38:49.989 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=84 2024-02-26T06:38:49.989 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=12 2024-02-26T06:38:49.990 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=1 2024-02-26T06:38:49.990 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:49.990 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:49.990 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=21 2024-02-26T06:38:49.990 DEBUG:tasks.check_counter:Getting stats from l 2024-02-26T06:38:49.991 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-26T06:38:50.232 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T06:38:50.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:49 smithi167 ceph-mon[36229]: pgmap v332: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:50.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:49 smithi090 ceph-mon[35681]: pgmap v332: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:50.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:49 smithi006 ceph-mon[31790]: pgmap v332: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:51.823 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-02-26T06:38:51.824 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T06:38:51.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 807, 2024-02-26T06:38:51.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 809, 2024-02-26T06:38:51.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 271097, 2024-02-26T06:38:51.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 250978, 2024-02-26T06:38:51.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.101772990, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.030856656, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.027022102, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.033069733, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 807, 2024-02-26T06:38:51.825 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.050185013, 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000062187 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T06:38:51.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1481, 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1655, 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 5229786, 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 997818, 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-02-26T06:38:51.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.117260906, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.040271956, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.030533444, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.036710741, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1652, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.075096765, 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000045458 2024-02-26T06:38:51.828 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 537792 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.829 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 280, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 250, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1970052, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 142048, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.052957861, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.015632406, 2024-02-26T06:38:51.830 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.042820390, 2024-02-26T06:38:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.005943429, 2024-02-26T06:38:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T06:38:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 250, 2024-02-26T06:38:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.016338952, 2024-02-26T06:38:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000065355 2024-02-26T06:38:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1968704, 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 141104 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.832 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3916, 2024-02-26T06:38:51.833 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.110510092, 2024-02-26T06:38:51.834 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000028220 2024-02-26T06:38:51.834 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.834 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.834 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-02-26T06:38:51.834 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-02-26T06:38:51.834 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-02-26T06:38:51.835 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-02-26T06:38:51.836 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 1223, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 8, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 1215, 2024-02-26T06:38:51.837 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 1211, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 14, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-02-26T06:38:51.838 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-02-26T06:38:51.839 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-02-26T06:38:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-02-26T06:38:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-02-26T06:38:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-02-26T06:38:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-02-26T06:38:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-02-26T06:38:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-02-26T06:38:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-02-26T06:38:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-02-26T06:38:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-02-26T06:38:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T06:38:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-02-26T06:38:51.849 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 2636, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-02-26T06:38:51.850 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 5, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 9046688, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 9046114, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.851 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 2636 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 549, 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 999, 2024-02-26T06:38:51.852 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 450, 2024-02-26T06:38:51.853 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 34, 2024-02-26T06:38:51.853 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 37, 2024-02-26T06:38:51.853 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 3, 2024-02-26T06:38:51.853 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 1223, 2024-02-26T06:38:51.853 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 1238, 2024-02-26T06:38:51.853 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 15, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 56376, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215420 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-02-26T06:38:51.854 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.855 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-02-26T06:38:51.856 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.857 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-02-26T06:38:51.858 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.858 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.858 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.858 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.858 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-02-26T06:38:51.858 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.859 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.860 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.861 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.862 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.862 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.862 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-02-26T06:38:51.862 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.862 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.863 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.863 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.863 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-02-26T06:38:51.863 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.863 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.863 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.864 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.865 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.866 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-02-26T06:38:51.867 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.868 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.869 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.870 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.871 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.872 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-02-26T06:38:51.872 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.872 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.872 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.872 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.872 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.873 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.874 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 2, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-02-26T06:38:51.875 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 4451, 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-02-26T06:38:51.876 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T06:38:51.877 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T06:38:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-02-26T06:38:51.879 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 218276, 2024-02-26T06:38:51.880 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 598, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-02-26T06:38:51.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8360, 2024-02-26T06:38:51.882 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 191, 2024-02-26T06:38:51.882 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T06:38:51.882 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-02-26T06:38:51.882 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-02-26T06:38:51.882 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-02-26T06:38:51.882 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 2128611, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 17998, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-02-26T06:38:51.883 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 2289, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 2289, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2289, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.413973906, 2024-02-26T06:38:51.884 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000617725 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2289, 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2290 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 2289, 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 2289, 2024-02-26T06:38:51.885 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1450, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 837, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-02-26T06:38:51.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-02-26T06:38:51.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-02-26T06:38:51.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-02-26T06:38:51.889 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-02-26T06:38:51.890 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-02-26T06:38:51.891 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 96, 2024-02-26T06:38:51.891 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-02-26T06:38:51.891 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 11, 2024-02-26T06:38:51.891 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 6, 2024-02-26T06:38:51.891 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 6, 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-02-26T06:38:51.892 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-02-26T06:38:51.893 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-02-26T06:38:51.894 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T06:38:51.894 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-02-26T06:38:51.894 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-02-26T06:38:51.894 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-02-26T06:38:51.894 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-02-26T06:38:51.894 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2564, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 7223213, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:51.895 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2564, 2024-02-26T06:38:51.896 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:51.896 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:51.896 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2564, 2024-02-26T06:38:51.896 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 7223213, 2024-02-26T06:38:51.896 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:51.896 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-02-26T06:38:51.897 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2289, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 4852389, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2289, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4852389, 2024-02-26T06:38:51.898 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:51.899 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2289, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2289, 2024-02-26T06:38:51.900 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2289, 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2289, 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-02-26T06:38:51.901 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:51.902 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55607855a960": { 2024-02-26T06:38:51.903 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-02-26T06:38:51.904 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-02-26T06:38:51.905 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-02-26T06:38:52.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:51 smithi167 ceph-mon[36229]: pgmap v333: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:52.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:51 smithi090 ceph-mon[35681]: pgmap v333: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:52.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:51 smithi006 ceph-mon[31790]: pgmap v333: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:52.637 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-02-26T06:38:52.637 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-02-26T06:38:52.637 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-02-26T06:38:52.637 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-02-26T06:38:52.637 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-02-26T06:38:52.638 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-02-26T06:38:52.638 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-02-26T06:38:52.638 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-02-26T06:38:52.638 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-26T06:38:52.650 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T06:38:52.650 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:38:52.650 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:38:52.713 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:38:52.713 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:38:52.771 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:38:53.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 -- 172.21.15.6:0/887028795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984103590 msgr2=0x7f0984103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 --2- 172.21.15.6:0/887028795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984103590 0x7f0984103970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0974009930 tx=0x7f097402bc20 comp rx=0 tx=0).stop 2024-02-26T06:38:53.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 -- 172.21.15.6:0/887028795 shutdown_connections 2024-02-26T06:38:53.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 --2- 172.21.15.6:0/887028795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984108660 0x7f0984110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 --2- 172.21.15.6:0/887028795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0984103f40 0x7f0984107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 --2- 172.21.15.6:0/887028795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984103590 0x7f0984103970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 -- 172.21.15.6:0/887028795 >> 172.21.15.6:0/887028795 conn(0x7f09840fee30 msgr2=0x7f0984101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:53.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 -- 172.21.15.6:0/887028795 shutdown_connections 2024-02-26T06:38:53.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.007+0000 7f0989200640 1 -- 172.21.15.6:0/887028795 wait complete. 2024-02-26T06:38:53.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.008+0000 7f0989200640 1 Processor -- start 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 -- start start 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 0x7f098419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984108660 0x7f098419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0984199830 0x7f0984199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098419a2e0 con 0x7f0984108660 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098419a450 con 0x7f0984103590 2024-02-26T06:38:53.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.009+0000 7f0989200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098419a5c0 con 0x7f0984199830 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984108660 0x7f098419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 0x7f098419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 0x7f098419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38370/0 (socket says 172.21.15.6:38370) 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 -- 172.21.15.6:0/1951236168 learned_addr learned my addr 172.21.15.6:0/1951236168 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0982800640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0984199830 0x7f0984199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 -- 172.21.15.6:0/1951236168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0984199830 msgr2=0x7f0984199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0984199830 0x7f0984199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 -- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984108660 msgr2=0x7f098419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984108660 0x7f098419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 -- 172.21.15.6:0/1951236168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0974009560 con 0x7f0984103590 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981400640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984108660 0x7f098419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.010+0000 7f0981e00640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 0x7f098419f6c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0974000c00 tx=0x7f097402c5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.011+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097402dea0 con 0x7f0984103590 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.011+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09740042d0 con 0x7f0984103590 2024-02-26T06:38:53.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.011+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974003a90 con 0x7f0984103590 2024-02-26T06:38:53.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.011+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09841a6510 con 0x7f0984103590 2024-02-26T06:38:53.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.011+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09841a69a0 con 0x7f0984103590 2024-02-26T06:38:53.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.012+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0984104a60 con 0x7f0984103590 2024-02-26T06:38:53.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.014+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f09740053d0 con 0x7f0984103590 2024-02-26T06:38:53.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.015+0000 7f097aa00640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f09680a22b0 0x7f09680a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.016+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f09740f0700 con 0x7f0984103590 2024-02-26T06:38:53.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.016+0000 7f0981400640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f09680a22b0 0x7f09680a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.016+0000 7f0981400640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f09680a22b0 0x7f09680a4770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0984102560 tx=0x7f096c005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:53.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.018+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09740b93f0 con 0x7f0984103590 2024-02-26T06:38:53.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.220+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f098410b890 con 0x7f0984103590 2024-02-26T06:38:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.222+0000 7f097aa00640 1 -- 172.21.15.6:0/1951236168 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+23131 (secure 0 0 0) 0x7f0974043020 con 0x7f0984103590 2024-02-26T06:38:53.222 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:38:53.222 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":96,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:38:10.069412+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1360414143":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/324687139":"2024-02-27T06:28:06.172431+0000","172.21.15.6:6801/324687139":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2856776212":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/3408505415":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/1854217832":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/64821628":"2024-02-27T06:28:06.172431+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:38:53.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f09680a22b0 msgr2=0x7f09680a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f09680a22b0 0x7f09680a4770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0984102560 tx=0x7f096c005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 msgr2=0x7f098419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 0x7f098419f6c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0974000c00 tx=0x7f097402c5e0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 shutdown_connections 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f09680a22b0 0x7f09680a4770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0984199830 0x7f0984199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0984108660 0x7f098419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 --2- 172.21.15.6:0/1951236168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0984103590 0x7f098419f6c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 >> 172.21.15.6:0/1951236168 conn(0x7f09840fee30 msgr2=0x7f0984101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:53.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 shutdown_connections 2024-02-26T06:38:53.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.225+0000 7f0989200640 1 -- 172.21.15.6:0/1951236168 wait complete. 2024-02-26T06:38:53.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T06:20:02.656992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T06:25:16.018157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T06:25:17.168594+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T06:38:53.247 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T06:38:53.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.466+0000 7fe702200640 1 -- 172.21.15.6:0/1749378448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc107f40 msgr2=0x7fe6fc114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.466+0000 7fe702200640 1 --2- 172.21.15.6:0/1749378448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc107f40 0x7fe6fc114430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009ca0 tx=0x7fe6f402c190 comp rx=0 tx=0).stop 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 -- 172.21.15.6:0/1749378448 shutdown_connections 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 --2- 172.21.15.6:0/1749378448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc107f40 0x7fe6fc114430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 --2- 172.21.15.6:0/1749378448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107580 0x7fe6fc107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 --2- 172.21.15.6:0/1749378448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc10c540 0x7fe6fc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 -- 172.21.15.6:0/1749378448 >> 172.21.15.6:0/1749378448 conn(0x7fe6fc076d80 msgr2=0x7fe6fc0791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 -- 172.21.15.6:0/1749378448 shutdown_connections 2024-02-26T06:38:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.467+0000 7fe702200640 1 -- 172.21.15.6:0/1749378448 wait complete. 2024-02-26T06:38:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 Processor -- start 2024-02-26T06:38:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 -- start start 2024-02-26T06:38:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 0x7fe6fc1a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107f40 0x7fe6fc1a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc10c540 0x7fe6fc19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc19e060 con 0x7fe6fc10c540 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc19e1d0 con 0x7fe6fc107580 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.468+0000 7fe702200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc19e390 con 0x7fe6fc107f40 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 0x7fe6fc1a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 0x7fe6fc1a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38382/0 (socket says 172.21.15.6:38382) 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe6fbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107f40 0x7fe6fc1a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 -- 172.21.15.6:0/897187918 learned_addr learned my addr 172.21.15.6:0/897187918 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe701800640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc10c540 0x7fe6fc19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 -- 172.21.15.6:0/897187918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107f40 msgr2=0x7fe6fc1a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107f40 0x7fe6fc1a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 -- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc10c540 msgr2=0x7fe6fc19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc10c540 0x7fe6fc19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 -- 172.21.15.6:0/897187918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4009900 con 0x7fe6fc107580 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe6fbe00640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107f40 0x7fe6fc1a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.469+0000 7fe700e00640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 0x7fe6fc1a3810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e9c0 tx=0x7fe6f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.470+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0009700 con 0x7fe6fc107580 2024-02-26T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.470+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc19e670 con 0x7fe6fc107580 2024-02-26T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.470+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f000cd80 con 0x7fe6fc107580 2024-02-26T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.470+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0010430 con 0x7fe6fc107580 2024-02-26T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.470+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc1aa8b0 con 0x7fe6fc107580 2024-02-26T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.471+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc04eee0 con 0x7fe6fc107580 2024-02-26T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.473+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe6f0003680 con 0x7fe6fc107580 2024-02-26T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.474+0000 7fe6f9600640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fe6e40a22b0 0x7fe6e40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.475+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fe6f0014070 con 0x7fe6fc107580 2024-02-26T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.475+0000 7fe6fbe00640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fe6e40a22b0 0x7fe6e40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.475+0000 7fe6fbe00640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fe6e40a22b0 0x7fe6e40a4770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0794f0 tx=0x7fe6e8005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.477+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6f0097030 con 0x7fe6fc107580 2024-02-26T06:38:53.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.679+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6fc10fb10 con 0x7fe6fc107580 2024-02-26T06:38:53.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.680+0000 7fe6f9600640 1 -- 172.21.15.6:0/897187918 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6fc10fb10 con 0x7fe6fc107580 2024-02-26T06:38:53.680 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-02-26T06:38:53.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fe6e40a22b0 msgr2=0x7fe6e40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fe6e40a22b0 0x7fe6e40a4770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0794f0 tx=0x7fe6e8005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:53.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 msgr2=0x7fe6fc1a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 0x7fe6fc1a3810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e9c0 tx=0x7fe6f000ee80 comp rx=0 tx=0).stop 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 shutdown_connections 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fe6e40a22b0 0x7fe6e40a4770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6fc10c540 0x7fe6fc19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc107f40 0x7fe6fc1a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 --2- 172.21.15.6:0/897187918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc107580 0x7fe6fc1a3810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.683+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 >> 172.21.15.6:0/897187918 conn(0x7fe6fc076d80 msgr2=0x7fe6fc077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.684+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 shutdown_connections 2024-02-26T06:38:53.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.684+0000 7fe702200640 1 -- 172.21.15.6:0/897187918 wait complete. 2024-02-26T06:38:53.703 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T06:38:53.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 -- 172.21.15.6:0/436229299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04940ffd80 msgr2=0x7f0494106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 --2- 172.21.15.6:0/436229299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04940ffd80 0x7f0494106de0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0480009890 tx=0x7f048002bc00 comp rx=0 tx=0).stop 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 -- 172.21.15.6:0/436229299 shutdown_connections 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 --2- 172.21.15.6:0/436229299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0494107510 0x7f049410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 --2- 172.21.15.6:0/436229299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04940ffd80 0x7f0494106de0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 --2- 172.21.15.6:0/436229299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ff460 0x7f04940ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.937+0000 7f049ac00640 1 -- 172.21.15.6:0/436229299 >> 172.21.15.6:0/436229299 conn(0x7f04940fb320 msgr2=0x7f04940fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.938+0000 7f049ac00640 1 -- 172.21.15.6:0/436229299 shutdown_connections 2024-02-26T06:38:53.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.938+0000 7f049ac00640 1 -- 172.21.15.6:0/436229299 wait complete. 2024-02-26T06:38:53.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 Processor -- start 2024-02-26T06:38:53.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 -- start start 2024-02-26T06:38:53.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04940ff460 0x7f0494100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ffd80 0x7f04941011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0494107510 0x7f04941016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941a5950 con 0x7f0494107510 2024-02-26T06:38:53.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941a5ac0 con 0x7f04940ffd80 2024-02-26T06:38:53.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.939+0000 7f049ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941a5c80 con 0x7f04940ff460 2024-02-26T06:38:53.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0499800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04940ff460 0x7f0494100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ffd80 0x7f04941011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0499800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04940ff460 0x7f0494100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.6:42758/0 (socket says 172.21.15.6:42758) 2024-02-26T06:38:53.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0499800640 1 -- 172.21.15.6:0/3094400852 learned_addr learned my addr 172.21.15.6:0/3094400852 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:53.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f049a200640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0494107510 0x7f04941016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 -- 172.21.15.6:0/3094400852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04940ff460 msgr2=0x7f0494100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04940ff460 0x7f0494100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 -- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0494107510 msgr2=0x7f04941016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:53.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0494107510 0x7f04941016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:53.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 -- 172.21.15.6:0/3094400852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009490 con 0x7f04940ffd80 2024-02-26T06:38:53.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.940+0000 7f0498e00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ffd80 0x7f04941011a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f048002c140 tx=0x7f04800344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:53.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.941+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048002c5a0 con 0x7f04940ffd80 2024-02-26T06:38:53.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.941+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04941a5f00 con 0x7f04940ffd80 2024-02-26T06:38:53.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.941+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f048002cbe0 con 0x7f04940ffd80 2024-02-26T06:38:53.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.941+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048003d3f0 con 0x7f04940ffd80 2024-02-26T06:38:53.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.941+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04941a6330 con 0x7f04940ffd80 2024-02-26T06:38:53.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.942+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049404eee0 con 0x7f04940ffd80 2024-02-26T06:38:53.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.945+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f048003b040 con 0x7f04940ffd80 2024-02-26T06:38:53.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.947+0000 7f0492000640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f047c0a22b0 0x7f047c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:53.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.947+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f04800efd20 con 0x7f04940ffd80 2024-02-26T06:38:53.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.947+0000 7f0499800640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f047c0a22b0 0x7f047c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:53.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.948+0000 7f0499800640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f047c0a22b0 0x7f047c0a4770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04880058c0 tx=0x7f0488009260 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:53.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:53.948+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04800b8ac0 con 0x7f04940ffd80 2024-02-26T06:38:54.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.152+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0494109500 con 0x7f04940ffd80 2024-02-26T06:38:54.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.153+0000 7f0492000640 1 -- 172.21.15.6:0/3094400852 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0480041020 con 0x7f04940ffd80 2024-02-26T06:38:54.153 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-02-26T06:38:54.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f047c0a22b0 msgr2=0x7f047c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f047c0a22b0 0x7f047c0a4770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04880058c0 tx=0x7f0488009260 comp rx=0 tx=0).stop 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ffd80 msgr2=0x7f04941011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ffd80 0x7f04941011a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f048002c140 tx=0x7f04800344c0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 shutdown_connections 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f047c0a22b0 0x7f047c0a4770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0494107510 0x7f04941016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04940ffd80 0x7f04941011a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 --2- 172.21.15.6:0/3094400852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04940ff460 0x7f0494100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 >> 172.21.15.6:0/3094400852 conn(0x7f04940fb320 msgr2=0x7f04940fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 shutdown_connections 2024-02-26T06:38:54.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.156+0000 7f049ac00640 1 -- 172.21.15.6:0/3094400852 wait complete. 2024-02-26T06:38:54.175 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T06:38:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:53 smithi167 ceph-mon[36229]: pgmap v334: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:54.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:53 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1951236168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:54.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:53 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/897187918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:38:54.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:53 smithi006 ceph-mon[31790]: pgmap v334: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:54.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:53 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1951236168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:54.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:53 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/897187918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:38:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:53 smithi090 ceph-mon[35681]: pgmap v334: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-26T06:38:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:53 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1951236168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:54.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:53 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/897187918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T06:38:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 -- 172.21.15.6:0/3047771271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c109590 msgr2=0x7f447c109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 --2- 172.21.15.6:0/3047771271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c109590 0x7f447c109970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f446c009930 tx=0x7f446c02bc20 comp rx=0 tx=0).stop 2024-02-26T06:38:54.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 -- 172.21.15.6:0/3047771271 shutdown_connections 2024-02-26T06:38:54.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 --2- 172.21.15.6:0/3047771271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c108240 0x7f447c114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 --2- 172.21.15.6:0/3047771271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f447c1078a0 0x7f447c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 --2- 172.21.15.6:0/3047771271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c109590 0x7f447c109970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 -- 172.21.15.6:0/3047771271 >> 172.21.15.6:0/3047771271 conn(0x7f447c077020 msgr2=0x7f447c105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:54.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 -- 172.21.15.6:0/3047771271 shutdown_connections 2024-02-26T06:38:54.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.406+0000 7f4483200640 1 -- 172.21.15.6:0/3047771271 wait complete. 2024-02-26T06:38:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.407+0000 7f4483200640 1 Processor -- start 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.407+0000 7f4483200640 1 -- start start 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4483200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 0x7f447c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4483200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c108240 0x7f447c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4483200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f447c109590 0x7f447c1a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4483200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a2de0 con 0x7f447c1078a0 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4483200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a2f50 con 0x7f447c108240 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4483200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a3110 con 0x7f447c109590 2024-02-26T06:38:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 0x7f447c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 0x7f447c19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42882/0 (socket says 172.21.15.6:42882) 2024-02-26T06:38:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 -- 172.21.15.6:0/253923436 learned_addr learned my addr 172.21.15.6:0/253923436 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447b400640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c108240 0x7f447c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f4481200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f447c109590 0x7f447c1a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 -- 172.21.15.6:0/253923436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f447c109590 msgr2=0x7f447c1a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f447c109590 0x7f447c1a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 -- 172.21.15.6:0/253923436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c108240 msgr2=0x7f447c19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c108240 0x7f447c19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.408+0000 7f447be00640 1 -- 172.21.15.6:0/253923436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c009560 con 0x7f447c1078a0 2024-02-26T06:38:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.409+0000 7f447be00640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 0x7f447c19df70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f446c000c00 tx=0x7f446c02c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.409+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0042d0 con 0x7f447c1078a0 2024-02-26T06:38:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.409+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f446c02de90 con 0x7f447c1078a0 2024-02-26T06:38:54.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.409+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c1a3390 con 0x7f447c1078a0 2024-02-26T06:38:54.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.409+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c003ab0 con 0x7f447c1078a0 2024-02-26T06:38:54.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.409+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1aac70 con 0x7f447c1078a0 2024-02-26T06:38:54.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.410+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c068450 con 0x7f447c1078a0 2024-02-26T06:38:54.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.412+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f446c003c10 con 0x7f447c1078a0 2024-02-26T06:38:54.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.414+0000 7f4478c00640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f44600a22b0 0x7f44600a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.414+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f446c0f0470 con 0x7f447c1078a0 2024-02-26T06:38:54.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.414+0000 7f447b400640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f44600a22b0 0x7f44600a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.415+0000 7f447b400640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f44600a22b0 0x7f44600a4770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f447c10ced0 tx=0x7f4468005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:54.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.419+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f446c0b9160 con 0x7f447c1078a0 2024-02-26T06:38:54.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.619+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f447c10fdd0 con 0x7f447c1078a0 2024-02-26T06:38:54.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.620+0000 7f4478c00640 1 -- 172.21.15.6:0/253923436 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f446c0bc820 con 0x7f447c1078a0 2024-02-26T06:38:54.619 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-02-26T06:38:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.623+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f44600a22b0 msgr2=0x7f44600a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f44600a22b0 0x7f44600a4770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f447c10ced0 tx=0x7f4468005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:54.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 msgr2=0x7f447c19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 0x7f447c19df70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f446c000c00 tx=0x7f446c02c660 comp rx=0 tx=0).stop 2024-02-26T06:38:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 shutdown_connections 2024-02-26T06:38:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f44600a22b0 0x7f44600a4770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f447c109590 0x7f447c1a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f447c108240 0x7f447c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 --2- 172.21.15.6:0/253923436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f447c1078a0 0x7f447c19df70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 >> 172.21.15.6:0/253923436 conn(0x7f447c077020 msgr2=0x7f447c0778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 shutdown_connections 2024-02-26T06:38:54.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.624+0000 7f4483200640 1 -- 172.21.15.6:0/253923436 wait complete. 2024-02-26T06:38:54.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T06:38:54.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 -- 172.21.15.6:0/382235253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 msgr2=0x7fab30104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 --2- 172.21.15.6:0/382235253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab30104cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fab18009900 tx=0x7fab1802bd30 comp rx=0 tx=0).stop 2024-02-26T06:38:54.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 -- 172.21.15.6:0/382235253 shutdown_connections 2024-02-26T06:38:54.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 --2- 172.21.15.6:0/382235253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301052f0 0x7fab30105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 --2- 172.21.15.6:0/382235253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab30104cd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 --2- 172.21.15.6:0/382235253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 0x7fab301002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 -- 172.21.15.6:0/382235253 >> 172.21.15.6:0/382235253 conn(0x7fab300fb960 msgr2=0x7fab300fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:54.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 -- 172.21.15.6:0/382235253 shutdown_connections 2024-02-26T06:38:54.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.859+0000 7fab35800640 1 -- 172.21.15.6:0/382235253 wait complete. 2024-02-26T06:38:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 Processor -- start 2024-02-26T06:38:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 -- start start 2024-02-26T06:38:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 0x7fab301a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab301a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301052f0 0x7fab3019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3019e090 con 0x7fab300ffef0 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3019e200 con 0x7fab30100810 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.861+0000 7fab35800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3019e3c0 con 0x7fab301052f0 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab2fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 0x7fab301a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab301a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab301a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:38426/0 (socket says 172.21.15.6:38426) 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 -- 172.21.15.6:0/4217084748 learned_addr learned my addr 172.21.15.6:0/4217084748 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab2fe00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 0x7fab301a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42906/0 (socket says 172.21.15.6:42906) 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab34e00640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301052f0 0x7fab3019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 -- 172.21.15.6:0/4217084748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301052f0 msgr2=0x7fab3019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301052f0 0x7fab3019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 -- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 msgr2=0x7fab301a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 0x7fab301a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.862+0000 7fab27e00640 1 -- 172.21.15.6:0/4217084748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009560 con 0x7fab30100810 2024-02-26T06:38:54.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.863+0000 7fab27e00640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab301a3dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab180045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:54.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.863+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1802cb50 con 0x7fab30100810 2024-02-26T06:38:54.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.863+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab1802ccb0 con 0x7fab30100810 2024-02-26T06:38:54.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.863+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3019e640 con 0x7fab30100810 2024-02-26T06:38:54.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.863+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18003b20 con 0x7fab30100810 2024-02-26T06:38:54.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.863+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab301aa930 con 0x7fab30100810 2024-02-26T06:38:54.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.864+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3004eee0 con 0x7fab30100810 2024-02-26T06:38:54.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.866+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fab18003c80 con 0x7fab30100810 2024-02-26T06:38:54.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.867+0000 7fab2d600640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fab140a22b0 0x7fab140a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:54.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.868+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fab180f0460 con 0x7fab30100810 2024-02-26T06:38:54.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.868+0000 7fab2fe00640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fab140a22b0 0x7fab140a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:54.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.869+0000 7fab2fe00640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fab140a22b0 0x7fab140a4770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab3019eb80 tx=0x7fab20005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:54.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:54.870+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab180b9150 con 0x7fab30100810 2024-02-26T06:38:55.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.086+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fab3019ef70 con 0x7fab30100810 2024-02-26T06:38:55.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.087+0000 7fab2d600640 1 -- 172.21.15.6:0/4217084748 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v96) v1 ==== 81+0+982 (secure 0 0 0) 0x7fab180423f0 con 0x7fab30100810 2024-02-26T06:38:55.087 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1360414143 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.087 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2348330080 2024-02-27T06:17:11.959803+0000 2024-02-26T06:38:55.087 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2587275670 2024-02-27T06:16:36.555104+0000 2024-02-26T06:38:55.087 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/1027725899 2024-02-27T06:16:36.555104+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/4032437212 2024-02-27T06:17:11.959803+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/483778690 2024-02-27T06:16:36.555104+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3180746931 2024-02-27T06:17:11.959803+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/1027725899 2024-02-27T06:16:36.555104+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2377177802 2024-02-27T06:17:11.959803+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/4032437212 2024-02-27T06:17:11.959803+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/324687139 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.088 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/324687139 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.089 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2856776212 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.089 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3408505415 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.089 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2705487638 2024-02-27T06:16:36.555104+0000 2024-02-26T06:38:55.089 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1854217832 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.089 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/64821628 2024-02-27T06:28:06.172431+0000 2024-02-26T06:38:55.090 INFO:teuthology.orchestra.run.smithi006.stderr:listed 17 entries 2024-02-26T06:38:55.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fab140a22b0 msgr2=0x7fab140a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fab140a22b0 0x7fab140a4770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab3019eb80 tx=0x7fab20005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 msgr2=0x7fab301a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab301a3dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab180045b0 comp rx=0 tx=0).stop 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 shutdown_connections 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fab140a22b0 0x7fab140a4770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301052f0 0x7fab3019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab30100810 0x7fab301a3dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:55.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 --2- 172.21.15.6:0/4217084748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab300ffef0 0x7fab301a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:55.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.090+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 >> 172.21.15.6:0/4217084748 conn(0x7fab300fb960 msgr2=0x7fab300fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:55.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.091+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 shutdown_connections 2024-02-26T06:38:55.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:55.091+0000 7fab35800640 1 -- 172.21.15.6:0/4217084748 wait complete. 2024-02-26T06:38:55.110 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T06:38:55.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.110 DEBUG:teuthology.orchestra.run.smithi006:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T06:38:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:54 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3094400852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:38:55.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:54 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/253923436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:38:55.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:54 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3094400852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:38:55.371 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:54 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/253923436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:38:55.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:38:55.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.375 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T06:38:55.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.433 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-02-26T06:38:55.470 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T06:38:55.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.471 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T06:38:55.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:54 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3094400852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T06:38:55.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:54 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/253923436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T06:38:55.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-02-26T06:38:55.716 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T06:38:55.716 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-02-26T06:38:55.716 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T06:38:55.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499600k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/6f69b4ea5573639cdc155362433a0ed434704c53a0bf39591266c9390ae0c5fe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/6f69b4ea5573639cdc155362433a0ed434704c53a0bf39591266c9390ae0c5fe/diff,workdir=/var/lib/containers/storage/overlay/6f69b4ea5573639cdc155362433a0ed434704c53a0bf39591266c9390ae0c5fe/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0061e57c693a4af963d67629eb8c3a11a5216ce0360dfd5502cfd13f02546ddc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/0061e57c693a4af963d67629eb8c3a11a5216ce0360dfd5502cfd13f02546ddc/diff,workdir=/var/lib/containers/storage/overlay/0061e57c693a4af963d67629eb8c3a11a5216ce0360dfd5502cfd13f02546ddc/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/9d0653d3ce2f6269622d1aa456b1dfef967b907c7b10e0ab787b1e364b834a57/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/9d0653d3ce2f6269622d1aa456b1dfef967b907c7b10e0ab787b1e364b834a57/diff,workdir=/var/lib/containers/storage/overlay/9d0653d3ce2f6269622d1aa456b1dfef967b907c7b10e0ab787b1e364b834a57/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f36f453a0be1ed9e4f43ea76932358fc97fae6da6478e1191329b0a56faf6309/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/f36f453a0be1ed9e4f43ea76932358fc97fae6da6478e1191329b0a56faf6309/diff,workdir=/var/lib/containers/storage/overlay/f36f453a0be1ed9e4f43ea76932358fc97fae6da6478e1191329b0a56faf6309/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/72fc8bb13c8aaa35c31c87a96a03dafe0a6a10e4cf550378ec0d8731717ac213/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/72fc8bb13c8aaa35c31c87a96a03dafe0a6a10e4cf550378ec0d8731717ac213/diff,workdir=/var/lib/containers/storage/overlay/72fc8bb13c8aaa35c31c87a96a03dafe0a6a10e4cf550378ec0d8731717ac213/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/11e0593cc42b8cd8f9972dfa19b3f14325ed8d0b012d6fdad443a92dbc51869f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/11e0593cc42b8cd8f9972dfa19b3f14325ed8d0b012d6fdad443a92dbc51869f/diff,workdir=/var/lib/containers/storage/overlay/11e0593cc42b8cd8f9972dfa19b3f14325ed8d0b012d6fdad443a92dbc51869f/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/71239f045754a2e30f85a1cdc660afb4715abdcd85099925e6506448d4b19d0c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/71239f045754a2e30f85a1cdc660afb4715abdcd85099925e6506448d4b19d0c/diff,workdir=/var/lib/containers/storage/overlay/71239f045754a2e30f85a1cdc660afb4715abdcd85099925e6506448d4b19d0c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/49ca2f71c0e42cdaf6d3b3f43b54df5e8cec3c78915c2b9f09bcb1416d55ebe8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/49ca2f71c0e42cdaf6d3b3f43b54df5e8cec3c78915c2b9f09bcb1416d55ebe8/diff,workdir=/var/lib/containers/storage/overlay/49ca2f71c0e42cdaf6d3b3f43b54df5e8cec3c78915c2b9f09bcb1416d55ebe8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/16e9d54b4dc1062b2d533a72fb6ea2ad7bf1893309a4a636c286e1014e79d568/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/16e9d54b4dc1062b2d533a72fb6ea2ad7bf1893309a4a636c286e1014e79d568/diff,workdir=/var/lib/containers/storage/overlay/16e9d54b4dc1062b2d533a72fb6ea2ad7bf1893309a4a636c286e1014e79d568/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/abfb87347e75fe51c8f2b07ae2c53feecf0881e3408f9a4763b18b2e8167263a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IQUFVSITOVRG5WX4QECNWRWRZ:/var/lib/containers/storage/overlay/l/QWXC3I74LIWKJRGPP5Q27NEONC,upperdir=/var/lib/containers/storage/overlay/abfb87347e75fe51c8f2b07ae2c53feecf0881e3408f9a4763b18b2e8167263a/diff,workdir=/var/lib/containers/storage/overlay/abfb87347e75fe51c8f2b07ae2c53feecf0881e3408f9a4763b18b2e8167263a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499600k,nr_inodes=819200,mode=755 0 0 2024-02-26T06:38:55.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.758 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-02-26T06:38:55.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T06:38:55.848 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-02-26T06:38:55.933 INFO:teuthology.orchestra.run.smithi006.stderr:Cannot find device "ceph-brx" 2024-02-26T06:38:55.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:38:55.937 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-26T06:38:55.948 INFO:tasks.cephadm:Teardown begin 2024-02-26T06:38:55.948 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T06:38:55.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:38:56.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.169+0000 7fede4c00640 1 -- 172.21.15.6:0/189482102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 msgr2=0x7fede0103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.169+0000 7fede4c00640 1 --2- 172.21.15.6:0/189482102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 0x7fede0103890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009860 tx=0x7fedd002bb70 comp rx=0 tx=0).stop 2024-02-26T06:38:56.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 -- 172.21.15.6:0/189482102 shutdown_connections 2024-02-26T06:38:56.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 --2- 172.21.15.6:0/189482102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 0x7fede0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 --2- 172.21.15.6:0/189482102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede0103e60 0x7fede0107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 --2- 172.21.15.6:0/189482102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 0x7fede0103890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 -- 172.21.15.6:0/189482102 >> 172.21.15.6:0/189482102 conn(0x7fede00fecb0 msgr2=0x7fede01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:56.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 -- 172.21.15.6:0/189482102 shutdown_connections 2024-02-26T06:38:56.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.170+0000 7fede4c00640 1 -- 172.21.15.6:0/189482102 wait complete. 2024-02-26T06:38:56.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 Processor -- start 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 -- start start 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 0x7fede019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 0x7fede019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede01996c0 0x7fede0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019a190 con 0x7fede0108580 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019a300 con 0x7fede01996c0 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.171+0000 7fede4c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019a4c0 con 0x7fede01034b0 2024-02-26T06:38:56.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7fedd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 0x7fede019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7fedd7e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 0x7fede019fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42926/0 (socket says 172.21.15.6:42926) 2024-02-26T06:38:56.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7fedd7e00640 1 -- 172.21.15.6:0/2096581482 learned_addr learned my addr 172.21.15.6:0/2096581482 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:56.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede01996c0 0x7fede0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddf400640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 0x7fede019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:56 smithi006 ceph-mon[31790]: pgmap v335: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:56.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:56 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/4217084748' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T06:38:56.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 -- 172.21.15.6:0/2096581482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 msgr2=0x7fede019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 0x7fede019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 -- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 msgr2=0x7fede019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 0x7fede019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 -- 172.21.15.6:0/2096581482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0009490 con 0x7fede01996c0 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.172+0000 7feddfe00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede01996c0 0x7fede0199b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009fd0 tx=0x7fedd800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.173+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd800ce20 con 0x7fede01996c0 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.173+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd8005020 con 0x7fede01996c0 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.173+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8010690 con 0x7fede01996c0 2024-02-26T06:38:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.173+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede01a6460 con 0x7fede01996c0 2024-02-26T06:38:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.173+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01a6940 con 0x7fede01996c0 2024-02-26T06:38:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.174+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede0104980 con 0x7fede01996c0 2024-02-26T06:38:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.176+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fedd8005500 con 0x7fede01996c0 2024-02-26T06:38:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.177+0000 7feddcc00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fedc40a22a0 0x7fedc40a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.178+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fedd8014070 con 0x7fede01996c0 2024-02-26T06:38:56.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.178+0000 7feddf400640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fedc40a22a0 0x7fedc40a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.179+0000 7feddf400640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fedc40a22a0 0x7fedc40a4760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedd002c180 tx=0x7fedd00056d0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:56.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.181+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedd8097350 con 0x7fede01996c0 2024-02-26T06:38:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:56 smithi090 ceph-mon[35681]: pgmap v335: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:56.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:56 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/4217084748' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T06:38:56.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.425+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fede010b7b0 con 0x7fede01996c0 2024-02-26T06:38:56.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.427+0000 7feddcc00640 1 -- 172.21.15.6:0/2096581482 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fedd8005190 con 0x7fede01996c0 2024-02-26T06:38:56.427 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:38:56.427 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:32:14.861538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":64,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:38:56.427 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 41 2024-02-26T06:38:56.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fedc40a22a0 msgr2=0x7fedc40a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fedc40a22a0 0x7fedc40a4760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedd002c180 tx=0x7fedd00056d0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede01996c0 msgr2=0x7fede0199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede01996c0 0x7fede0199b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009fd0 tx=0x7fedd800ef10 comp rx=0 tx=0).stop 2024-02-26T06:38:56.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 shutdown_connections 2024-02-26T06:38:56.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fedc40a22a0 0x7fedc40a4760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede01996c0 0x7fede0199b40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0108580 0x7fede019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 --2- 172.21.15.6:0/2096581482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede01034b0 0x7fede019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 >> 172.21.15.6:0/2096581482 conn(0x7fede00fecb0 msgr2=0x7fede0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:56.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 shutdown_connections 2024-02-26T06:38:56.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.430+0000 7fede4c00640 1 -- 172.21.15.6:0/2096581482 wait complete. 2024-02-26T06:38:56.449 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T06:38:56.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:56 smithi167 ceph-mon[36229]: pgmap v335: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:56.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:56 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/4217084748' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T06:38:56.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.676+0000 7f6dbba00640 1 -- 172.21.15.6:0/3385968870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db4108670 msgr2=0x7f6db4110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 --2- 172.21.15.6:0/3385968870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db4108670 0x7f6db4110240 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009cd0 tx=0x7f6dac02c0e0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 -- 172.21.15.6:0/3385968870 shutdown_connections 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 --2- 172.21.15.6:0/3385968870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db4108670 0x7f6db4110240 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 --2- 172.21.15.6:0/3385968870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4103f50 0x7f6db4107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 --2- 172.21.15.6:0/3385968870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db41035a0 0x7f6db4103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 -- 172.21.15.6:0/3385968870 >> 172.21.15.6:0/3385968870 conn(0x7f6db40fee40 msgr2=0x7f6db4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 -- 172.21.15.6:0/3385968870 shutdown_connections 2024-02-26T06:38:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.677+0000 7f6dbba00640 1 -- 172.21.15.6:0/3385968870 wait complete. 2024-02-26T06:38:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 Processor -- start 2024-02-26T06:38:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 -- start start 2024-02-26T06:38:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db41035a0 0x7f6db419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4108670 0x7f6db419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db4199820 0x7f6db4199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db419a2d0 con 0x7f6db41035a0 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db419a440 con 0x7f6db4199820 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.678+0000 7f6dbba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db419a5b0 con 0x7f6db4108670 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db41035a0 0x7f6db419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db4199820 0x7f6db4199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db41035a0 0x7f6db419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42946/0 (socket says 172.21.15.6:42946) 2024-02-26T06:38:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9000640 1 -- 172.21.15.6:0/2371109024 learned_addr learned my addr 172.21.15.6:0/2371109024 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db3e00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4108670 0x7f6db419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9a00640 1 -- 172.21.15.6:0/2371109024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4108670 msgr2=0x7f6db419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9a00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4108670 0x7f6db419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9a00640 1 -- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db41035a0 msgr2=0x7f6db419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9a00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db41035a0 0x7f6db419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.679+0000 7f6db9a00640 1 -- 172.21.15.6:0/2371109024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009900 con 0x7f6db4199820 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.680+0000 7f6db9a00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db4199820 0x7f6db4199cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6dac037040 tx=0x7f6dac034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.680+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0328b0 con 0x7f6db4199820 2024-02-26T06:38:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.680+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dac003e50 con 0x7f6db4199820 2024-02-26T06:38:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.680+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db41a6500 con 0x7f6db4199820 2024-02-26T06:38:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.680+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac004780 con 0x7f6db4199820 2024-02-26T06:38:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.680+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db41a6930 con 0x7f6db4199820 2024-02-26T06:38:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.681+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4104a70 con 0x7f6db4199820 2024-02-26T06:38:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.683+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f6dac00a040 con 0x7f6db4199820 2024-02-26T06:38:56.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.685+0000 7f6db1600640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6d9c0a2230 0x7f6d9c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:56.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.685+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f6dac0f10c0 con 0x7f6db4199820 2024-02-26T06:38:56.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.685+0000 7f6db9000640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6d9c0a2230 0x7f6d9c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:56.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.686+0000 7f6db9000640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6d9c0a2230 0x7f6d9c0a46f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009c90 tx=0x7f6da8005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:56.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.687+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dac0b9db0 con 0x7f6db4199820 2024-02-26T06:38:56.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.927+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6db410b940 con 0x7f6db4199820 2024-02-26T06:38:56.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.928+0000 7f6db1600640 1 -- 172.21.15.6:0/2371109024 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f6dac0bd470 con 0x7f6db4199820 2024-02-26T06:38:56.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:38:56.928 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14376,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1064493742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1064493742},{"type":"v1","addr":"172.21.15.6:6835","nonce":1064493742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14439,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/1183641688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":1183641688},{"type":"v1","addr":"172.21.15.6:6839","nonce":1183641688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T06:25:17.657061+0000","modified":"2024-02-26T06:32:14.861538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24569,"mds_2":24544,"mds_3":24530,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/66780761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":66780761},{"type":"v1","addr":"172.21.15.167:6833","nonce":66780761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/707593181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":707593181},{"type":"v1","addr":"172.21.15.90:6833","nonce":707593181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/234881158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":234881158},{"type":"v1","addr":"172.21.15.6:6837","nonce":234881158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/679395830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":679395830},{"type":"v1","addr":"172.21.15.90:6835","nonce":679395830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/2489701568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2489701568},{"type":"v1","addr":"172.21.15.167:6835","nonce":2489701568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6837/30308127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":30308127},{"type":"v1","addr":"172.21.15.167:6837","nonce":30308127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.90:6837/3613516978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3613516978},{"type":"v1","addr":"172.21.15.90:6837","nonce":3613516978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":64,"addr":"172.21.15.167:6839/3988004767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3988004767},{"type":"v1","addr":"172.21.15.167:6839","nonce":3988004767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.6:6841/3799445989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":3799445989},{"type":"v1","addr":"172.21.15.6:6841","nonce":3799445989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6839/137401855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":137401855},{"type":"v1","addr":"172.21.15.90:6839","nonce":137401855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T06:38:56.929 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 41 2024-02-26T06:38:56.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6d9c0a2230 msgr2=0x7f6d9c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6d9c0a2230 0x7f6d9c0a46f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009c90 tx=0x7f6da8005f90 comp rx=0 tx=0).stop 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db4199820 msgr2=0x7f6db4199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db4199820 0x7f6db4199cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6dac037040 tx=0x7f6dac034a90 comp rx=0 tx=0).stop 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 shutdown_connections 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f6d9c0a2230 0x7f6d9c0a46f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6db4199820 0x7f6db4199cd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4108670 0x7f6db419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 --2- 172.21.15.6:0/2371109024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6db41035a0 0x7f6db419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 >> 172.21.15.6:0/2371109024 conn(0x7f6db40fee40 msgr2=0x7f6db4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:56.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 shutdown_connections 2024-02-26T06:38:56.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:56.931+0000 7f6dbba00640 1 -- 172.21.15.6:0/2371109024 wait complete. 2024-02-26T06:38:56.948 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T06:38:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 -- 172.21.15.6:0/3728938524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 msgr2=0x7f85a8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 --2- 172.21.15.6:0/3728938524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a8107ba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8590009930 tx=0x7f859002bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 -- 172.21.15.6:0/3728938524 shutdown_connections 2024-02-26T06:38:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 --2- 172.21.15.6:0/3728938524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a81080e0 0x7f85a81145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 --2- 172.21.15.6:0/3728938524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a8107ba0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 --2- 172.21.15.6:0/3728938524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a810d730 0x7f85a810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 -- 172.21.15.6:0/3728938524 >> 172.21.15.6:0/3728938524 conn(0x7f85a8076ea0 msgr2=0x7f85a80792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 -- 172.21.15.6:0/3728938524 shutdown_connections 2024-02-26T06:38:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.165+0000 7f85ae400640 1 -- 172.21.15.6:0/3728938524 wait complete. 2024-02-26T06:38:57.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.166+0000 7f85ae400640 1 Processor -- start 2024-02-26T06:38:57.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 -- start start 2024-02-26T06:38:57.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a81080e0 0x7f85a819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a810d730 0x7f85a81a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a81a2c90 con 0x7f85a8107720 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a81a2e00 con 0x7f85a81080e0 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85ae400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a81a2fc0 con 0x7f85a810d730 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85a7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85a7400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59588/0 (socket says 172.21.15.6:59588) 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85a7400640 1 -- 172.21.15.6:0/1204379921 learned_addr learned my addr 172.21.15.6:0/1204379921 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f859ea00640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a81080e0 0x7f85a819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.167+0000 7f85a7e00640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a810d730 0x7f85a81a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f859ea00640 1 -- 172.21.15.6:0/1204379921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a810d730 msgr2=0x7f85a81a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f859ea00640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a810d730 0x7f85a81a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f859ea00640 1 -- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 msgr2=0x7f85a819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f859ea00640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f859ea00640 1 -- 172.21.15.6:0/1204379921 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0009490 con 0x7f85a81080e0 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f859ea00640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a81080e0 0x7f85a819e360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8590009230 tx=0x7f859002c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.168+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85900042d0 con 0x7f85a81080e0 2024-02-26T06:38:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.169+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8590004430 con 0x7f85a81080e0 2024-02-26T06:38:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.169+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590003bd0 con 0x7f85a81080e0 2024-02-26T06:38:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.169+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8590009560 con 0x7f85a81080e0 2024-02-26T06:38:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.169+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a81aa940 con 0x7f85a81080e0 2024-02-26T06:38:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.170+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a8068450 con 0x7f85a81080e0 2024-02-26T06:38:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.172+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8590003d30 con 0x7f85a81080e0 2024-02-26T06:38:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.173+0000 7f85a4c00640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f858c0a22b0 0x7f858c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.173+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7f85900effe0 con 0x7f85a81080e0 2024-02-26T06:38:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.173+0000 7f85a7400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f858c0a22b0 0x7f858c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.174+0000 7f85a7400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f858c0a22b0 0x7f858c0a4770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8598009fa0 tx=0x7f8598009500 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:57.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.175+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85900b8cd0 con 0x7f85a81080e0 2024-02-26T06:38:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2096581482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:38:57.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2371109024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:38:57.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.373+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85a81178c0 con 0x7f85a81080e0 2024-02-26T06:38:57.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.375+0000 7f85a4c00640 1 -- 172.21.15.6:0/1204379921 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+23131 (secure 0 0 0) 0x7f85900bc390 con 0x7f85a81080e0 2024-02-26T06:38:57.375 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-02-26T06:38:57.375 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":96,"fsid":"5e92f702-d46e-11ee-95c3-87774f69a715","created":"2024-02-26T06:16:02.582083+0000","modified":"2024-02-26T06:38:10.069412+0000","last_up_change":"2024-02-26T06:24:15.850850+0000","last_in_change":"2024-02-26T06:23:54.799664+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T06:20:02.656992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T06:25:16.018157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T06:25:17.168594+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d33ce97-4574-4b39-a887-734262e13d40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6803","nonce":2652019447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6805","nonce":2652019447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6809","nonce":2652019447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":2652019447},{"type":"v1","addr":"172.21.15.6:6807","nonce":2652019447}]},"public_addr":"172.21.15.6:6803/2652019447","cluster_addr":"172.21.15.6:6805/2652019447","heartbeat_back_addr":"172.21.15.6:6809/2652019447","heartbeat_front_addr":"172.21.15.6:6807/2652019447","state":["exists","up"]},{"osd":1,"uuid":"71372d37-1d3f-46c1-abc4-3a33d0daf773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6801","nonce":2679347906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6803","nonce":2679347906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6807","nonce":2679347906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2679347906},{"type":"v1","addr":"172.21.15.90:6805","nonce":2679347906}]},"public_addr":"172.21.15.90:6801/2679347906","cluster_addr":"172.21.15.90:6803/2679347906","heartbeat_back_addr":"172.21.15.90:6807/2679347906","heartbeat_front_addr":"172.21.15.90:6805/2679347906","state":["exists","up"]},{"osd":2,"uuid":"f8b7b566-c20f-4c25-83c7-3cc834dcbaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6801","nonce":1414245392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6803","nonce":1414245392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6807","nonce":1414245392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1414245392},{"type":"v1","addr":"172.21.15.167:6805","nonce":1414245392}]},"public_addr":"172.21.15.167:6801/1414245392","cluster_addr":"172.21.15.167:6803/1414245392","heartbeat_back_addr":"172.21.15.167:6807/1414245392","heartbeat_front_addr":"172.21.15.167:6805/1414245392","state":["exists","up"]},{"osd":3,"uuid":"513bca2a-d4e4-4934-8826-3b204caa92b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6811","nonce":3201157546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6813","nonce":3201157546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6817","nonce":3201157546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3201157546},{"type":"v1","addr":"172.21.15.6:6815","nonce":3201157546}]},"public_addr":"172.21.15.6:6811/3201157546","cluster_addr":"172.21.15.6:6813/3201157546","heartbeat_back_addr":"172.21.15.6:6817/3201157546","heartbeat_front_addr":"172.21.15.6:6815/3201157546","state":["exists","up"]},{"osd":4,"uuid":"e2ca8afc-575f-4cdd-8310-2c62fba98c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6809","nonce":108774388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6811","nonce":108774388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6815","nonce":108774388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":108774388},{"type":"v1","addr":"172.21.15.90:6813","nonce":108774388}]},"public_addr":"172.21.15.90:6809/108774388","cluster_addr":"172.21.15.90:6811/108774388","heartbeat_back_addr":"172.21.15.90:6815/108774388","heartbeat_front_addr":"172.21.15.90:6813/108774388","state":["exists","up"]},{"osd":5,"uuid":"d5e79bb3-2052-4a0d-8b43-d6bf454109b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6809","nonce":3058964685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6811","nonce":3058964685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6815","nonce":3058964685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3058964685},{"type":"v1","addr":"172.21.15.167:6813","nonce":3058964685}]},"public_addr":"172.21.15.167:6809/3058964685","cluster_addr":"172.21.15.167:6811/3058964685","heartbeat_back_addr":"172.21.15.167:6815/3058964685","heartbeat_front_addr":"172.21.15.167:6813/3058964685","state":["exists","up"]},{"osd":6,"uuid":"2f9e1699-08b0-422a-9875-5a840fa6c47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6819","nonce":643219798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6821","nonce":643219798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6825","nonce":643219798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":643219798},{"type":"v1","addr":"172.21.15.6:6823","nonce":643219798}]},"public_addr":"172.21.15.6:6819/643219798","cluster_addr":"172.21.15.6:6821/643219798","heartbeat_back_addr":"172.21.15.6:6825/643219798","heartbeat_front_addr":"172.21.15.6:6823/643219798","state":["exists","up"]},{"osd":7,"uuid":"3cfbd6e5-1fd5-4cdd-ae2e-1dd05fe4d635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6817","nonce":2240633136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6819","nonce":2240633136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6823","nonce":2240633136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2240633136},{"type":"v1","addr":"172.21.15.90:6821","nonce":2240633136}]},"public_addr":"172.21.15.90:6817/2240633136","cluster_addr":"172.21.15.90:6819/2240633136","heartbeat_back_addr":"172.21.15.90:6823/2240633136","heartbeat_front_addr":"172.21.15.90:6821/2240633136","state":["exists","up"]},{"osd":8,"uuid":"863e848b-67fd-4bc7-82c5-b3b69df061b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6817","nonce":734768771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6819","nonce":734768771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6823","nonce":734768771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":734768771},{"type":"v1","addr":"172.21.15.167:6821","nonce":734768771}]},"public_addr":"172.21.15.167:6817/734768771","cluster_addr":"172.21.15.167:6819/734768771","heartbeat_back_addr":"172.21.15.167:6823/734768771","heartbeat_front_addr":"172.21.15.167:6821/734768771","state":["exists","up"]},{"osd":9,"uuid":"6ee193e3-93b3-47fb-84b4-39a14c4a060f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6827","nonce":1514059134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6829","nonce":1514059134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6833","nonce":1514059134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1514059134},{"type":"v1","addr":"172.21.15.6:6831","nonce":1514059134}]},"public_addr":"172.21.15.6:6827/1514059134","cluster_addr":"172.21.15.6:6829/1514059134","heartbeat_back_addr":"172.21.15.6:6833/1514059134","heartbeat_front_addr":"172.21.15.6:6831/1514059134","state":["exists","up"]},{"osd":10,"uuid":"d0f22982-ab74-4fe3-b273-a2d992a543e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6825","nonce":1676953993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6827","nonce":1676953993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6831","nonce":1676953993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1676953993},{"type":"v1","addr":"172.21.15.90:6829","nonce":1676953993}]},"public_addr":"172.21.15.90:6825/1676953993","cluster_addr":"172.21.15.90:6827/1676953993","heartbeat_back_addr":"172.21.15.90:6831/1676953993","heartbeat_front_addr":"172.21.15.90:6829/1676953993","state":["exists","up"]},{"osd":11,"uuid":"f631c7a6-a02d-434a-9f64-3b60310f0222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6825","nonce":1697184190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6827","nonce":1697184190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6831","nonce":1697184190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1697184190},{"type":"v1","addr":"172.21.15.167:6829","nonce":1697184190}]},"public_addr":"172.21.15.167:6825/1697184190","cluster_addr":"172.21.15.167:6827/1697184190","heartbeat_back_addr":"172.21.15.167:6831/1697184190","heartbeat_front_addr":"172.21.15.167:6829/1697184190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:09.147050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:34.056181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:19:58.423073+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:31.535024+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:20:57.062384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:21.931943+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:21:56.669330+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:22.714624+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:22:47.748935+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:22.017645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:23:48.025339+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T06:24:13.078789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1360414143":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2348330080":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/2587275670":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6801/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:6800/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:0/483778690":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/3180746931":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/1027725899":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/2377177802":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6801/4032437212":"2024-02-27T06:17:11.959803+0000","172.21.15.6:6800/324687139":"2024-02-27T06:28:06.172431+0000","172.21.15.6:6801/324687139":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2856776212":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/3408505415":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/2705487638":"2024-02-27T06:16:36.555104+0000","172.21.15.6:0/1854217832":"2024-02-27T06:28:06.172431+0000","172.21.15.6:0/64821628":"2024-02-27T06:28:06.172431+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f858c0a22b0 msgr2=0x7f858c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f858c0a22b0 0x7f858c0a4770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8598009fa0 tx=0x7f8598009500 comp rx=0 tx=0).stop 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a81080e0 msgr2=0x7f85a819e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a81080e0 0x7f85a819e360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8590009230 tx=0x7f859002c5d0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 shutdown_connections 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f858c0a22b0 0x7f858c0a4770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a810d730 0x7f85a81a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a81080e0 0x7f85a819e360 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 --2- 172.21.15.6:0/1204379921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85a8107720 0x7f85a819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.378+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 >> 172.21.15.6:0/1204379921 conn(0x7f85a8076ea0 msgr2=0x7f85a80776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:57.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.379+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 shutdown_connections 2024-02-26T06:38:57.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.379+0000 7f85ae400640 1 -- 172.21.15.6:0/1204379921 wait complete. 2024-02-26T06:38:57.393 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2096581482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:38:57.393 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2371109024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:38:57.397 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T06:38:57.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2096581482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:38:57.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2371109024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T06:38:57.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.627+0000 7fb117e00640 1 -- 172.21.15.6:0/889389381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 msgr2=0x7fb118103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.627+0000 7fb117e00640 1 --2- 172.21.15.6:0/889389381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb118103790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb108009930 tx=0x7fb10802bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:57.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 -- 172.21.15.6:0/889389381 shutdown_connections 2024-02-26T06:38:57.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 --2- 172.21.15.6:0/889389381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb118108480 0x7fb118110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 --2- 172.21.15.6:0/889389381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118103d60 0x7fb118107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 --2- 172.21.15.6:0/889389381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb118103790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 -- 172.21.15.6:0/889389381 >> 172.21.15.6:0/889389381 conn(0x7fb1180fec30 msgr2=0x7fb118101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:57.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 -- 172.21.15.6:0/889389381 shutdown_connections 2024-02-26T06:38:57.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.628+0000 7fb117e00640 1 -- 172.21.15.6:0/889389381 wait complete. 2024-02-26T06:38:57.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 Processor -- start 2024-02-26T06:38:57.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 -- start start 2024-02-26T06:38:57.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb11819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118108480 0x7fb11819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb118199640 0x7fb118199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11819a110 con 0x7fb1181033b0 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11819a280 con 0x7fb118108480 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.629+0000 7fb117e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11819a440 con 0x7fb118199640 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb11819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb11819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59602/0 (socket says 172.21.15.6:59602) 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118108480 0x7fb11819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116a00640 1 -- 172.21.15.6:0/2740480172 learned_addr learned my addr 172.21.15.6:0/2740480172 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:57.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb117400640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb118199640 0x7fb118199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 -- 172.21.15.6:0/2740480172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb118199640 msgr2=0x7fb118199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb118199640 0x7fb118199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 -- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 msgr2=0x7fb11819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb11819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 -- 172.21.15.6:0/2740480172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009490 con 0x7fb118108480 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116a00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb11819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.630+0000 7fb116000640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118108480 0x7fb11819fac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e9c0 tx=0x7fb10400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:57.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.631+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400ccc0 con 0x7fb118108480 2024-02-26T06:38:57.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.631+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb10400ce20 con 0x7fb118108480 2024-02-26T06:38:57.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.631+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb110009840 con 0x7fb118108480 2024-02-26T06:38:57.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.631+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104010430 con 0x7fb118108480 2024-02-26T06:38:57.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.631+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb108009560 con 0x7fb118108480 2024-02-26T06:38:57.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.632+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb118104880 con 0x7fb118108480 2024-02-26T06:38:57.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.635+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb104003f60 con 0x7fb118108480 2024-02-26T06:38:57.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.636+0000 7fb10f400640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fb0fc0a22b0 0x7fb0fc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:57.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.636+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8593+0+0 (secure 0 0 0) 0x7fb104014070 con 0x7fb118108480 2024-02-26T06:38:57.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.637+0000 7fb116a00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fb0fc0a22b0 0x7fb0fc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:57.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.638+0000 7fb116a00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fb0fc0a22b0 0x7fb0fc0a4770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb108000c00 tx=0x7fb1080058d0 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:57.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.641+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1040970e0 con 0x7fb118108480 2024-02-26T06:38:57.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.884+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb11810b6b0 con 0x7fb118108480 2024-02-26T06:38:57.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.896+0000 7fb10f400640 1 -- 172.21.15.6:0/2740480172 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb10409a7a0 con 0x7fb118108480 2024-02-26T06:38:57.895 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fb0fc0a22b0 msgr2=0x7fb0fc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fb0fc0a22b0 0x7fb0fc0a4770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb108000c00 tx=0x7fb1080058d0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118108480 msgr2=0x7fb11819fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118108480 0x7fb11819fac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e9c0 tx=0x7fb10400edc0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 shutdown_connections 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7fb0fc0a22b0 0x7fb0fc0a4770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb118199640 0x7fb118199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb118108480 0x7fb11819fac0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.899+0000 7fb117e00640 1 --2- 172.21.15.6:0/2740480172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1181033b0 0x7fb11819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.900+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 >> 172.21.15.6:0/2740480172 conn(0x7fb1180fec30 msgr2=0x7fb118100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.900+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 shutdown_connections 2024-02-26T06:38:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:57.900+0000 7fb117e00640 1 -- 172.21.15.6:0/2740480172 wait complete. 2024-02-26T06:38:57.917 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T06:38:58.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 -- 172.21.15.6:0/3224427202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073e70 msgr2=0x7f903810d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:58.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 --2- 172.21.15.6:0/3224427202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073e70 0x7f903810d460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9024009860 tx=0x7f902402bc10 comp rx=0 tx=0).stop 2024-02-26T06:38:58.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 -- 172.21.15.6:0/3224427202 shutdown_connections 2024-02-26T06:38:58.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 --2- 172.21.15.6:0/3224427202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903810d9a0 0x7f903810fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 --2- 172.21.15.6:0/3224427202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073e70 0x7f903810d460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 --2- 172.21.15.6:0/3224427202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9038073550 0x7f9038073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 -- 172.21.15.6:0/3224427202 >> 172.21.15.6:0/3224427202 conn(0x7f90380fcbf0 msgr2=0x7f90380ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:58.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.107+0000 7f903ea00640 1 -- 172.21.15.6:0/3224427202 shutdown_connections 2024-02-26T06:38:58.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.108+0000 7f903ea00640 1 -- 172.21.15.6:0/3224427202 wait complete. 2024-02-26T06:38:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.108+0000 7f903ea00640 1 Processor -- start 2024-02-26T06:38:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 -- start start 2024-02-26T06:38:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073550 0x7f903819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9038073e70 0x7f903819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 0x7f9038199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038199eb0 con 0x7f903810d9a0 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903819a020 con 0x7f9038073550 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903819a1e0 con 0x7f9038073e70 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 0x7f9038199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 0x7f9038199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59672/0 (socket says 172.21.15.6:59672) 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f903ca00640 1 -- 172.21.15.6:0/3424857437 learned_addr learned my addr 172.21.15.6:0/3424857437 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f9037400640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9038073e70 0x7f903819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.109+0000 7f9037e00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073550 0x7f903819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ca00640 1 -- 172.21.15.6:0/3424857437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9038073e70 msgr2=0x7f903819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ca00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9038073e70 0x7f903819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ca00640 1 -- 172.21.15.6:0/3424857437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073550 msgr2=0x7f903819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ca00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073550 0x7f903819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ca00640 1 -- 172.21.15.6:0/3424857437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028009560 con 0x7f903810d9a0 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ca00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 0x7f9038199860 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f903000eb10 tx=0x7f903000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903000ce80 con 0x7f903810d9a0 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9024009490 con 0x7f903810d9a0 2024-02-26T06:38:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9030005020 con 0x7f903810d9a0 2024-02-26T06:38:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030010700 con 0x7f903810d9a0 2024-02-26T06:38:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.110+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90381a6570 con 0x7f903810d9a0 2024-02-26T06:38:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.111+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038074c00 con 0x7f903810d9a0 2024-02-26T06:38:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.113+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f90300054e0 con 0x7f903810d9a0 2024-02-26T06:38:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.115+0000 7f9034c00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f901c0a2230 0x7f901c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.115+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8979+0+0 (secure 0 0 0) 0x7f9030014070 con 0x7f903810d9a0 2024-02-26T06:38:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.115+0000 7f9037e00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f901c0a2230 0x7f901c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:58.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.116+0000 7f9037e00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f901c0a2230 0x7f901c0a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f903819a9a0 tx=0x7f9028009260 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:58.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.119+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9030097410 con 0x7f903810d9a0 2024-02-26T06:38:58.172 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d[72525]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.172 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d[72525]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.172 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-mds[72536]: -- 172.21.15.6:0/868025184 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ebccf841a0 con 0x55ebccf5b000 2024-02-26T06:38:58.173 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d[72525]: starting mds.d at 2024-02-26T06:38:58.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:38:57.886+0000 7efe17a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:38:58.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: pgmap v336: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:58.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1204379921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:58.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2740480172' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T06:38:58.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:58 smithi006 ceph-mon[31790]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-26T06:38:58.175 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a[73477]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.175 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a[73477]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.175 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-mds[73494]: -- 172.21.15.6:0/2955907401 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e902eb21a0 con 0x55e902e88c00 2024-02-26T06:38:58.175 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:38:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a[73477]: starting mds.a at 2024-02-26T06:38:58.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: pgmap v336: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1204379921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2740480172' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-26T06:38:58.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T06:38:58.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T06:38:58.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:58 smithi167 ceph-mon[36229]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-26T06:38:58.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l[66837]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l[66837]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-mds[66841]: -- 172.21.15.167:0/1557444473 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d22125c1a0 con 0x55d221233000 2024-02-26T06:38:58.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l[66837]: starting mds.l at 2024-02-26T06:38:58.247 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c[67451]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.247 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c[67451]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.247 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-mds[67455]: -- 172.21.15.167:0/2096846689 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c68545e1a0 con 0x55c685434c00 2024-02-26T06:38:58.247 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c[67451]: starting mds.c at 2024-02-26T06:38:58.248 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f[67145]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.248 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f[67145]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.248 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-mds[67149]: -- 172.21.15.167:0/1275913234 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55742720c1a0 con 0x5574271e2800 2024-02-26T06:38:58.248 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f[67145]: starting mds.f at 2024-02-26T06:38:58.248 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.249 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.249 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-mds[67779]: -- 172.21.15.167:0/1052373770 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b63a3061a0 con 0x55b63a2dd000 2024-02-26T06:38:58.249 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:38:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: starting mds.i at 2024-02-26T06:38:58.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.352+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9038108b20 con 0x7f903810d9a0 2024-02-26T06:38:58.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h[67019]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h[67019]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mds[67023]: -- 172.21.15.90:0/1150135055 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c9323401a0 con 0x55c932317000 2024-02-26T06:38:58.365 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h[67019]: starting mds.h at 2024-02-26T06:38:58.366 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e[66711]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.366 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e[66711]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.366 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mds[66715]: -- 172.21.15.90:0/99497269 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fc13ce9080 con 0x55fc13cb3000 2024-02-26T06:38:58.366 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mds[66715]: -- 172.21.15.90:0/99497269 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fc13cdc1a0 con 0x55fc13cb3000 2024-02-26T06:38:58.366 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e[66711]: starting mds.e at 2024-02-26T06:38:58.366 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k[66389]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.367 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k[66389]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.367 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mds[66393]: -- 172.21.15.90:0/2050059773 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a39ee6c1a0 con 0x55a39ee42c00 2024-02-26T06:38:58.367 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k[66389]: starting mds.k at 2024-02-26T06:38:58.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: pgmap v336: 97 pgs: 97 active+clean; 76 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-26T06:38:58.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1204379921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T06:38:58.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2740480172' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T06:38:58.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:58 smithi090 ceph-mon[35681]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-26T06:38:58.369 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: ignoring --setuser ceph since I am not root 2024-02-26T06:38:58.369 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: ignoring --setgroup ceph since I am not root 2024-02-26T06:38:58.369 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-mds[67339]: -- 172.21.15.90:0/2859577133 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5630814201a0 con 0x5630813f7000 2024-02-26T06:38:58.369 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:38:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: starting mds.b at 2024-02-26T06:38:58.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.892+0000 7f9034c00640 1 -- 172.21.15.6:0/3424857437 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f903009aad0 con 0x7f903810d9a0 2024-02-26T06:38:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f901c0a2230 msgr2=0x7f901c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f901c0a2230 0x7f901c0a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f903819a9a0 tx=0x7f9028009260 comp rx=0 tx=0).stop 2024-02-26T06:38:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 msgr2=0x7f9038199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 0x7f9038199860 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f903000eb10 tx=0x7f903000c2d0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 shutdown_connections 2024-02-26T06:38:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f901c0a2230 0x7f901c0a46f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903810d9a0 0x7f9038199860 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9038073e70 0x7f903819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 --2- 172.21.15.6:0/3424857437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9038073550 0x7f903819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:58.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.896+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 >> 172.21.15.6:0/3424857437 conn(0x7f90380fcbf0 msgr2=0x7f90380fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:58.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.897+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 shutdown_connections 2024-02-26T06:38:58.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:58.897+0000 7f903ea00640 1 -- 172.21.15.6:0/3424857437 wait complete. 2024-02-26T06:38:58.923 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T06:38:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 -- 172.21.15.6:0/3373990073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc741078a0 msgr2=0x7efc74107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 --2- 172.21.15.6:0/3373990073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc741078a0 0x7efc74107d00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc60009930 tx=0x7efc6002bc30 comp rx=0 tx=0).stop 2024-02-26T06:38:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 -- 172.21.15.6:0/3373990073 shutdown_connections 2024-02-26T06:38:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 --2- 172.21.15.6:0/3373990073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc74108240 0x7efc74114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 --2- 172.21.15.6:0/3373990073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc741078a0 0x7efc74107d00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:59.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 --2- 172.21.15.6:0/3373990073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc74109590 0x7efc74109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 -- 172.21.15.6:0/3373990073 >> 172.21.15.6:0/3373990073 conn(0x7efc74077020 msgr2=0x7efc74105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:38:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 -- 172.21.15.6:0/3373990073 shutdown_connections 2024-02-26T06:38:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.151+0000 7efc7ac00640 1 -- 172.21.15.6:0/3373990073 wait complete. 2024-02-26T06:38:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.152+0000 7efc7ac00640 1 Processor -- start 2024-02-26T06:38:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.152+0000 7efc7ac00640 1 -- start start 2024-02-26T06:38:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc7ac00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc741078a0 0x7efc7419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc7ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc74108240 0x7efc7419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc7ac00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 0x7efc741a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc7ac00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a2cf0 con 0x7efc74109590 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc7ac00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a2e60 con 0x7efc741078a0 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc7ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a3020 con 0x7efc74108240 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc78c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 0x7efc741a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc78c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 0x7efc741a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59686/0 (socket says 172.21.15.6:59686) 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc78c00640 1 -- 172.21.15.6:0/1938686883 learned_addr learned my addr 172.21.15.6:0/1938686883 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc73400640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc74108240 0x7efc7419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.153+0000 7efc73e00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc741078a0 0x7efc7419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc78c00640 1 -- 172.21.15.6:0/1938686883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc74108240 msgr2=0x7efc7419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc78c00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc74108240 0x7efc7419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc78c00640 1 -- 172.21.15.6:0/1938686883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc741078a0 msgr2=0x7efc7419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc78c00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc741078a0 0x7efc7419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc78c00640 1 -- 172.21.15.6:0/1938686883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc60009560 con 0x7efc74109590 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc78c00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 0x7efc741a26f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00ea60 tx=0x7efc6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00ce40 con 0x7efc74109590 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc741a3300 con 0x7efc74109590 2024-02-26T06:38:59.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc6c005020 con 0x7efc74109590 2024-02-26T06:38:59.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c010690 con 0x7efc74109590 2024-02-26T06:38:59.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.154+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc741aab80 con 0x7efc74109590 2024-02-26T06:38:59.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.156+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc74068450 con 0x7efc74109590 2024-02-26T06:38:59.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.158+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7efc6c005500 con 0x7efc74109590 2024-02-26T06:38:59.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.159+0000 7efc70c00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7efc580a2230 0x7efc580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:38:59.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.159+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8979+0+0 (secure 0 0 0) 0x7efc6c014070 con 0x7efc74109590 2024-02-26T06:38:59.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.159+0000 7efc73e00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7efc580a2230 0x7efc580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:38:59.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.160+0000 7efc73e00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7efc580a2230 0x7efc580a46f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc64009bd0 tx=0x7efc64005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:38:59.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.164+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc6c097360 con 0x7efc74109590 2024-02-26T06:38:59.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3424857437' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: Cluster is now healthy 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/3424857437' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6836/1542467006,v1:172.21.15.90:6839/1542467006] up:boot 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6834/574334479,v1:172.21.15.90:6838/574334479] up:boot 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6832/882194313,v1:172.21.15.90:6835/882194313] up:boot 2024-02-26T06:38:59.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.90:6833/2624749607,v1:172.21.15.90:6837/2624749607] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6833/3202864949,v1:172.21.15.167:6835/3202864949] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6832/1513460721,v1:172.21.15.167:6834/1513460721] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6837/3951922242,v1:172.21.15.167:6839/3951922242] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.167:6836/1125760304,v1:172.21.15.167:6838/1125760304] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6836/4200902140,v1:172.21.15.6:6837/4200902140] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: mds.? [v2:172.21.15.6:6840/1717191791,v1:172.21.15.6:6841/1717191791] up:boot 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: fsmap 12 up:standby 2024-02-26T06:38:59.173 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:38:59.174 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:38:59 smithi006 ceph-mon[31790]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:38:59.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:38:59.363+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efc7410fdd0 con 0x7efc74109590 2024-02-26T06:38:59.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3424857437' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:38:59.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T06:38:59.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T06:38:59.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: Cluster is now healthy 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/3424857437' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6836/1542467006,v1:172.21.15.90:6839/1542467006] up:boot 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6834/574334479,v1:172.21.15.90:6838/574334479] up:boot 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6832/882194313,v1:172.21.15.90:6835/882194313] up:boot 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.90:6833/2624749607,v1:172.21.15.90:6837/2624749607] up:boot 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6833/3202864949,v1:172.21.15.167:6835/3202864949] up:boot 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6832/1513460721,v1:172.21.15.167:6834/1513460721] up:boot 2024-02-26T06:38:59.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6837/3951922242,v1:172.21.15.167:6839/3951922242] up:boot 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.167:6836/1125760304,v1:172.21.15.167:6838/1125760304] up:boot 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6836/4200902140,v1:172.21.15.6:6837/4200902140] up:boot 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: mds.? [v2:172.21.15.6:6840/1717191791,v1:172.21.15.6:6841/1717191791] up:boot 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: fsmap 12 up:standby 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:38:59.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:38:59.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:38:59.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:38:59.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:38:59.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:38:59.368 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:38:59 smithi090 ceph-mon[35681]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:38:59.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3424857437' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T06:38:59.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T06:38:59.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T06:38:59.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: Cluster is now healthy 2024-02-26T06:38:59.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/3424857437' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T06:38:59.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6836/1542467006,v1:172.21.15.90:6839/1542467006] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6834/574334479,v1:172.21.15.90:6838/574334479] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6832/882194313,v1:172.21.15.90:6835/882194313] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.90:6833/2624749607,v1:172.21.15.90:6837/2624749607] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6833/3202864949,v1:172.21.15.167:6835/3202864949] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6832/1513460721,v1:172.21.15.167:6834/1513460721] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6837/3951922242,v1:172.21.15.167:6839/3951922242] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.167:6836/1125760304,v1:172.21.15.167:6838/1125760304] up:boot 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T06:38:59.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6836/4200902140,v1:172.21.15.6:6837/4200902140] up:boot 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: mds.? [v2:172.21.15.6:6840/1717191791,v1:172.21.15.6:6841/1717191791] up:boot 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: fsmap 12 up:standby 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T06:38:59.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T06:38:59.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T06:38:59.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T06:38:59.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:38:59 smithi167 ceph-mon[36229]: from='mgr.15330 172.21.15.6:0/3285829440' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T06:39:00.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.039+0000 7efc70c00640 1 -- 172.21.15.6:0/1938686883 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7efc6c005190 con 0x7efc74109590 2024-02-26T06:39:00.039 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2024-02-26T06:39:00.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7efc580a2230 msgr2=0x7efc580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:00.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7efc580a2230 0x7efc580a46f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc64009bd0 tx=0x7efc64005f90 comp rx=0 tx=0).stop 2024-02-26T06:39:00.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 msgr2=0x7efc741a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:00.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 0x7efc741a26f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00ea60 tx=0x7efc6c00ef20 comp rx=0 tx=0).stop 2024-02-26T06:39:00.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 shutdown_connections 2024-02-26T06:39:00.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7efc580a2230 0x7efc580a46f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc74109590 0x7efc741a26f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc74108240 0x7efc7419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 --2- 172.21.15.6:0/1938686883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc741078a0 0x7efc7419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.043+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 >> 172.21.15.6:0/1938686883 conn(0x7efc74077020 msgr2=0x7efc7410af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:00.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.044+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 shutdown_connections 2024-02-26T06:39:00.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.045+0000 7efc7ac00640 1 -- 172.21.15.6:0/1938686883 wait complete. 2024-02-26T06:39:00.069 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T06:39:00.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.276+0000 7f5d77e00640 1 -- 172.21.15.6:0/278067713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 msgr2=0x7f5d78110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:00.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.276+0000 7f5d77e00640 1 --2- 172.21.15.6:0/278067713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d78110050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009cd0 tx=0x7f5d7002c0e0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 -- 172.21.15.6:0/278067713 shutdown_connections 2024-02-26T06:39:00.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 --2- 172.21.15.6:0/278067713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d78110050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 --2- 172.21.15.6:0/278067713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78103d60 0x7f5d78107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 --2- 172.21.15.6:0/278067713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d781033b0 0x7f5d78103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 -- 172.21.15.6:0/278067713 >> 172.21.15.6:0/278067713 conn(0x7f5d780febf0 msgr2=0x7f5d78101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:00.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 -- 172.21.15.6:0/278067713 shutdown_connections 2024-02-26T06:39:00.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.277+0000 7f5d77e00640 1 -- 172.21.15.6:0/278067713 wait complete. 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 Processor -- start 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 -- start start 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d781033b0 0x7f5d7819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d7819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78199650 0x7f5d78199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819a040 con 0x7f5d78108480 2024-02-26T06:39:00.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819a1b0 con 0x7f5d78199650 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.278+0000 7f5d77e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819a370 con 0x7f5d781033b0 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d76000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d7819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d76000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d7819f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59710/0 (socket says 172.21.15.6:59710) 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78199650 0x7f5d78199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d76000640 1 -- 172.21.15.6:0/2352298654 learned_addr learned my addr 172.21.15.6:0/2352298654 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d76a00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d781033b0 0x7f5d7819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d77400640 1 -- 172.21.15.6:0/2352298654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d781033b0 msgr2=0x7f5d7819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:00.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d77400640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d781033b0 0x7f5d7819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d77400640 1 -- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 msgr2=0x7f5d7819f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:00.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d77400640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d7819f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:00.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.279+0000 7f5d77400640 1 -- 172.21.15.6:0/2352298654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009900 con 0x7f5d78199650 2024-02-26T06:39:00.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.280+0000 7f5d77400640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78199650 0x7f5d78199b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5d70037040 tx=0x7f5d700325e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:00.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.280+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70034e20 con 0x7f5d78199650 2024-02-26T06:39:00.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.280+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d781a6310 con 0x7f5d78199650 2024-02-26T06:39:00.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.280+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d70003e50 con 0x7f5d78199650 2024-02-26T06:39:00.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.280+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70004790 con 0x7f5d78199650 2024-02-26T06:39:00.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.280+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781a6790 con 0x7f5d78199650 2024-02-26T06:39:00.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.281+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7804eee0 con 0x7f5d78199650 2024-02-26T06:39:00.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.284+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f5d7000a040 con 0x7f5d78199650 2024-02-26T06:39:00.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.285+0000 7f5d6f400640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5d5c0a2230 0x7f5d5c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T06:39:00.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.285+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f5d700f0780 con 0x7f5d78199650 2024-02-26T06:39:00.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.285+0000 7f5d76a00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5d5c0a2230 0x7f5d5c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T06:39:00.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.286+0000 7f5d76a00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5d5c0a2230 0x7f5d5c0a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009bd0 tx=0x7f5d68005f90 comp rx=0 tx=0).ready entity=mgr.15330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T06:39:00.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.289+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d700b9520 con 0x7f5d78199650 2024-02-26T06:39:00.321 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:00 smithi006 ceph-mon[31790]: pgmap v338: 97 pgs: 97 active+clean; 76 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-02-26T06:39:00.322 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:00 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1938686883' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:00 smithi090 ceph-mon[35681]: pgmap v338: 97 pgs: 97 active+clean; 76 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-02-26T06:39:00.365 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:00 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1938686883' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:00.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:00.489+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d7810b710 con 0x7f5d78199650 2024-02-26T06:39:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:00 smithi167 ceph-mon[36229]: pgmap v338: 97 pgs: 97 active+clean; 76 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-02-26T06:39:00.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:00 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1938686883' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.068+0000 7f5d6f400640 1 -- 172.21.15.6:0/2352298654 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d700bcbe0 con 0x7f5d78199650 2024-02-26T06:39:01.067 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2024-02-26T06:39:01.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.072+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5d5c0a2230 msgr2=0x7f5d5c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:01.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.072+0000 7f5d77e00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5d5c0a2230 0x7f5d5c0a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009bd0 tx=0x7f5d68005f90 comp rx=0 tx=0).stop 2024-02-26T06:39:01.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.072+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78199650 msgr2=0x7f5d78199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T06:39:01.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.072+0000 7f5d77e00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78199650 0x7f5d78199b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5d70037040 tx=0x7f5d700325e0 comp rx=0 tx=0).stop 2024-02-26T06:39:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.073+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 shutdown_connections 2024-02-26T06:39:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.073+0000 7f5d77e00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:6800/3331986569,v1:172.21.15.6:6801/3331986569] conn(0x7f5d5c0a2230 0x7f5d5c0a46f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.073+0000 7f5d77e00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d78199650 0x7f5d78199b00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.073+0000 7f5d77e00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d78108480 0x7f5d7819f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:01.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.073+0000 7f5d77e00640 1 --2- 172.21.15.6:0/2352298654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d781033b0 0x7f5d7819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T06:39:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.074+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 >> 172.21.15.6:0/2352298654 conn(0x7f5d780febf0 msgr2=0x7f5d78100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T06:39:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.074+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 shutdown_connections 2024-02-26T06:39:01.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-02-26T06:39:01.074+0000 7f5d77e00640 1 -- 172.21.15.6:0/2352298654 wait complete. 2024-02-26T06:39:01.103 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:39:01.114 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:01 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/1938686883' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T06:39:01.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:01 smithi090 ceph-mon[35681]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T06:39:01.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:01 smithi090 ceph-mon[35681]: from='client.? 172.21.15.6:0/2352298654' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.115 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:01 smithi090 ceph-mon[35681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:39:01.210 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:39:01.243 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-26T06:39:01.243 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:39:01.299 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:39:01.314 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T06:39:01.330 INFO:tasks.cephadm:Stopping all daemons... 2024-02-26T06:39:01.331 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-26T06:39:01.331 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a 2024-02-26T06:39:01.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/1938686883' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T06:39:01.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 ceph-mon[31790]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T06:39:01.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 ceph-mon[31790]: from='client.? 172.21.15.6:0/2352298654' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 ceph-mon[31790]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:01 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/1938686883' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T06:39:01.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:01 smithi167 ceph-mon[36229]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T06:39:01.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:01 smithi167 ceph-mon[36229]: from='client.? 172.21.15.6:0/2352298654' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:01 smithi167 ceph-mon[36229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T06:39:01.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 systemd[1]: Stopping Ceph mon.a for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:39:01.956+0000 7efe1e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a[31786]: 2024-02-26T06:39:01.956+0000 7efe1e800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-26T06:39:02.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:02 smithi006 podman[99313]: 2024-02-26 06:39:02.142092197 +0000 UTC m=+0.361651221 container stop daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-26T06:39:02.423 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:02 smithi006 podman[99313]: 2024-02-26 06:39:02.168915123 +0000 UTC m=+0.388474098 container died daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:39:02.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:02 smithi006 podman[99313]: 2024-02-26 06:39:02.582331807 +0000 UTC m=+0.801890828 container cleanup daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:39:02.833 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:02 smithi006 bash[99313]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a 2024-02-26T06:39:03.171 INFO:journalctl@ceph.mon.a.smithi006.stdout:Feb 26 06:39:02 smithi006 podman[99325]: 2024-02-26 06:39:02.833231674 +0000 UTC m=+0.667324958 container remove daf24c1bbbd0358fca3ff2a3ea907a080cd3db369159fe4e5dab90b7db861e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-26T06:39:03.263 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.a.service' 2024-02-26T06:39:03.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:03.419 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-26T06:39:03.420 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-26T06:39:03.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.b 2024-02-26T06:39:03.718 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:03 smithi090 systemd[1]: Stopping Ceph mon.b for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:04.027 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-b[35677]: 2024-02-26T06:39:03.718+0000 7f1f434c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:04.027 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-b[35677]: 2024-02-26T06:39:03.718+0000 7f1f434c3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-26T06:39:04.027 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:03 smithi090 podman[73261]: 2024-02-26 06:39:03.739201347 +0000 UTC m=+0.120517508 container died 09d413e3f1140c433e9782e3a30cbf9ee1de2625293616dfe4a4656802c01b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-02-26T06:39:04.319 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:04 smithi090 podman[73261]: 2024-02-26 06:39:04.119270394 +0000 UTC m=+0.500586542 container cleanup 09d413e3f1140c433e9782e3a30cbf9ee1de2625293616dfe4a4656802c01b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:39:04.320 INFO:journalctl@ceph.mon.b.smithi090.stdout:Feb 26 06:39:04 smithi090 bash[73261]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-b 2024-02-26T06:39:04.450 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.b.service' 2024-02-26T06:39:04.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:04.496 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-26T06:39:04.496 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-26T06:39:04.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.c 2024-02-26T06:39:04.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:04 smithi167 systemd[1]: Stopping Ceph mon.c for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:05.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c[36222]: 2024-02-26T06:39:04.798+0000 7f2208ffc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:05.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c[36222]: 2024-02-26T06:39:04.798+0000 7f2208ffc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-26T06:39:05.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:04 smithi167 podman[73777]: 2024-02-26 06:39:04.900978071 +0000 UTC m=+0.202936892 container died fd386e6b9e0d77facb10a6d47cea64626ba9df382a144cfa216c53c7afc3fbfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T06:39:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:05 smithi167 podman[73777]: 2024-02-26 06:39:05.290880888 +0000 UTC m=+0.592839709 container cleanup fd386e6b9e0d77facb10a6d47cea64626ba9df382a144cfa216c53c7afc3fbfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T06:39:05.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Feb 26 06:39:05 smithi167 bash[73777]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mon-c 2024-02-26T06:39:05.643 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mon.c.service' 2024-02-26T06:39:05.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:05.694 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-26T06:39:05.694 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-26T06:39:05.694 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x 2024-02-26T06:39:06.171 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:05 smithi006 systemd[1]: Stopping Ceph mgr.x for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:06.672 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:39:06.326+0000 7f242a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:06.673 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x[32030]: 2024-02-26T06:39:06.326+0000 7f242a600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-26T06:39:06.673 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:06 smithi006 podman[99432]: 2024-02-26 06:39:06.412087453 +0000 UTC m=+0.220881950 container stop 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:39:06.673 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:06 smithi006 podman[99432]: 2024-02-26 06:39:06.441432147 +0000 UTC m=+0.250226648 container died 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-26T06:39:07.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:06 smithi006 podman[99432]: 2024-02-26 06:39:06.920102676 +0000 UTC m=+0.728897207 container cleanup 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-26T06:39:07.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:06 smithi006 bash[99432]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x 2024-02-26T06:39:07.421 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Feb 26 06:39:07 smithi006 podman[99446]: 2024-02-26 06:39:07.154691712 +0000 UTC m=+0.720307883 container remove 521a74fa328718b1e52674614179a949d75055d8a15a257a59b4600b013c03f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-x, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T06:39:07.672 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.x.service' 2024-02-26T06:39:07.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:07.801 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-26T06:39:07.801 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-26T06:39:07.801 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.y 2024-02-26T06:39:08.115 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:39:07 smithi090 systemd[1]: Stopping Ceph mgr.y for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:08.406 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:39:08 smithi090 podman[73384]: 2024-02-26 06:39:08.144559606 +0000 UTC m=+0.144093149 container died 8bed9c8b972df0354b50ac446bef9995ddec47f8c49bfb2f8191ee302dfd94d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:39:08.776 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:39:08 smithi090 podman[73384]: 2024-02-26 06:39:08.509045118 +0000 UTC m=+0.508578672 container cleanup 8bed9c8b972df0354b50ac446bef9995ddec47f8c49bfb2f8191ee302dfd94d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-26T06:39:08.776 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Feb 26 06:39:08 smithi090 bash[73384]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-y 2024-02-26T06:39:08.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.y.service' 2024-02-26T06:39:08.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:08.964 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-26T06:39:08.964 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-26T06:39:08.964 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.z 2024-02-26T06:39:09.245 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:39:09 smithi167 systemd[1]: Stopping Ceph mgr.z for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:09.579 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:39:09 smithi167 podman[73914]: 2024-02-26 06:39:09.292477901 +0000 UTC m=+0.146514536 container died f4db46f9f89bbfbfd2250f70f6ef1a31ff73d3a74612367ceeaec8764b28ef82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-26T06:39:09.994 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:39:09 smithi167 podman[73914]: 2024-02-26 06:39:09.654047675 +0000 UTC m=+0.508084272 container cleanup f4db46f9f89bbfbfd2250f70f6ef1a31ff73d3a74612367ceeaec8764b28ef82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-26T06:39:09.995 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Feb 26 06:39:09 smithi167 bash[73914]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mgr-z 2024-02-26T06:39:10.220 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mgr.z.service' 2024-02-26T06:39:10.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:10.259 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-26T06:39:10.260 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-26T06:39:10.260 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.0 2024-02-26T06:39:10.671 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:10 smithi006 systemd[1]: Stopping Ceph osd.0 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:11.172 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0[46314]: 2024-02-26T06:39:10.898+0000 7fc284000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:11.172 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0[46314]: 2024-02-26T06:39:10.898+0000 7fc284000700 -1 osd.0 99 *** Got signal Terminated *** 2024-02-26T06:39:11.172 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0[46314]: 2024-02-26T06:39:10.898+0000 7fc284000700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:39:16.667 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:16 smithi006 podman[99554]: 2024-02-26 06:39:16.253645132 +0000 UTC m=+5.537716138 container died 5b81f99350a9a3c132894130bf23a1f1fc70140086a39e913d46b39c13b7b057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:39:16.921 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:16 smithi006 podman[99554]: 2024-02-26 06:39:16.849745796 +0000 UTC m=+6.133816835 container cleanup 5b81f99350a9a3c132894130bf23a1f1fc70140086a39e913d46b39c13b7b057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-26T06:39:16.921 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:16 smithi006 bash[99554]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0 2024-02-26T06:39:17.421 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:17 smithi006 podman[99571]: 2024-02-26 06:39:17.142597334 +0000 UTC m=+0.873700565 container remove 5b81f99350a9a3c132894130bf23a1f1fc70140086a39e913d46b39c13b7b057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-26T06:39:18.056 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:17 smithi006 podman[99662]: 2024-02-26 06:39:17.623886701 +0000 UTC m=+0.038074657 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:39:18.056 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:17 smithi006 podman[99662]: 2024-02-26 06:39:17.752159322 +0000 UTC m=+0.166347297 container create 1a31839948e0c4e0eab7e9b938c5c7e66285b754d4335a9160c60855bf2a06eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-26T06:39:18.057 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:18 smithi006 podman[99662]: 2024-02-26 06:39:18.056139821 +0000 UTC m=+0.470327853 container init 1a31839948e0c4e0eab7e9b938c5c7e66285b754d4335a9160c60855bf2a06eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True) 2024-02-26T06:39:18.421 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:18 smithi006 podman[99662]: 2024-02-26 06:39:18.0690957 +0000 UTC m=+0.483283682 container start 1a31839948e0c4e0eab7e9b938c5c7e66285b754d4335a9160c60855bf2a06eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-02-26T06:39:18.422 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:18 smithi006 podman[99662]: 2024-02-26 06:39:18.163022449 +0000 UTC m=+0.577210434 container attach 1a31839948e0c4e0eab7e9b938c5c7e66285b754d4335a9160c60855bf2a06eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-26T06:39:18.777 INFO:journalctl@ceph.osd.0.smithi006.stdout:Feb 26 06:39:18 smithi006 podman[99662]: 2024-02-26 06:39:18.521883797 +0000 UTC m=+0.936071782 container died 1a31839948e0c4e0eab7e9b938c5c7e66285b754d4335a9160c60855bf2a06eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-26T06:39:19.236 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.0.service' 2024-02-26T06:39:19.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:19.337 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-26T06:39:19.338 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-26T06:39:19.338 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.1 2024-02-26T06:39:19.631 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:19 smithi090 systemd[1]: Stopping Ceph osd.1 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:20.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1[40326]: 2024-02-26T06:39:19.631+0000 7fb6ad03f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:20.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1[40326]: 2024-02-26T06:39:19.631+0000 7fb6ad03f700 -1 osd.1 99 *** Got signal Terminated *** 2024-02-26T06:39:20.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1[40326]: 2024-02-26T06:39:19.631+0000 7fb6ad03f700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:39:25.224 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:24 smithi090 podman[73506]: 2024-02-26 06:39:24.950065288 +0000 UTC m=+5.422802707 container died 556178e51b85bf537c37ed1db0e5583e01d16ed50045a54e5b428901a339e12e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-26T06:39:25.551 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:25 smithi090 podman[73506]: 2024-02-26 06:39:25.342130409 +0000 UTC m=+5.814867828 container cleanup 556178e51b85bf537c37ed1db0e5583e01d16ed50045a54e5b428901a339e12e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T06:39:25.551 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:25 smithi090 bash[73506]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1 2024-02-26T06:39:25.848 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:25 smithi090 podman[73535]: 2024-02-26 06:39:25.551890078 +0000 UTC m=+0.602941590 container remove 556178e51b85bf537c37ed1db0e5583e01d16ed50045a54e5b428901a339e12e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:39:26.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:25 smithi090 podman[73630]: 2024-02-26 06:39:25.751706973 +0000 UTC m=+0.017781271 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:39:26.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:25 smithi090 podman[73630]: 2024-02-26 06:39:25.970067853 +0000 UTC m=+0.236142136 container create 85389e4bb98ad071db4a97bb156a55bf370b3a3030352d6457e133e20bfcb4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True) 2024-02-26T06:39:26.491 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:26 smithi090 podman[73630]: 2024-02-26 06:39:26.198778147 +0000 UTC m=+0.464852452 container init 85389e4bb98ad071db4a97bb156a55bf370b3a3030352d6457e133e20bfcb4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-26T06:39:26.491 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:26 smithi090 podman[73630]: 2024-02-26 06:39:26.202392644 +0000 UTC m=+0.468466924 container start 85389e4bb98ad071db4a97bb156a55bf370b3a3030352d6457e133e20bfcb4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-26T06:39:26.492 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:26 smithi090 podman[73630]: 2024-02-26 06:39:26.281444085 +0000 UTC m=+0.547518380 container attach 85389e4bb98ad071db4a97bb156a55bf370b3a3030352d6457e133e20bfcb4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-26T06:39:26.834 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:26 smithi090 podman[73630]: 2024-02-26 06:39:26.491727427 +0000 UTC m=+0.757801711 container died 85389e4bb98ad071db4a97bb156a55bf370b3a3030352d6457e133e20bfcb4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T06:39:27.079 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.1.service' 2024-02-26T06:39:27.086 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:27 smithi090 podman[73657]: 2024-02-26 06:39:27.062137024 +0000 UTC m=+0.564869878 container remove 85389e4bb98ad071db4a97bb156a55bf370b3a3030352d6457e133e20bfcb4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-26T06:39:27.086 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:27 smithi090 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-26T06:39:27.086 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:27 smithi090 systemd[1]: Stopped Ceph osd.1 for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:39:27.086 INFO:journalctl@ceph.osd.1.smithi090.stdout:Feb 26 06:39:27 smithi090 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.1.service: Consumed 14.333s CPU time. 2024-02-26T06:39:27.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:27.125 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-26T06:39:27.125 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-26T06:39:27.126 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.2 2024-02-26T06:39:27.433 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:27 smithi167 systemd[1]: Stopping Ceph osd.2 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:27.745 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2[41410]: 2024-02-26T06:39:27.432+0000 7f260a230700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:27.745 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2[41410]: 2024-02-26T06:39:27.432+0000 7f260a230700 -1 osd.2 99 *** Got signal Terminated *** 2024-02-26T06:39:27.745 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2[41410]: 2024-02-26T06:39:27.432+0000 7f260a230700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:39:33.192 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:32 smithi167 podman[74037]: 2024-02-26 06:39:32.917627784 +0000 UTC m=+5.598450635 container died 033f9dfc0e40e82c95059fcc3e68d9b8aec11cbf7372be194fe9c438bc383e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2) 2024-02-26T06:39:33.494 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:33 smithi167 podman[74037]: 2024-02-26 06:39:33.31627599 +0000 UTC m=+5.997098843 container cleanup 033f9dfc0e40e82c95059fcc3e68d9b8aec11cbf7372be194fe9c438bc383e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T06:39:33.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:33 smithi167 bash[74037]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2 2024-02-26T06:39:33.935 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:33 smithi167 podman[74055]: 2024-02-26 06:39:33.576049195 +0000 UTC m=+0.657363530 container remove 033f9dfc0e40e82c95059fcc3e68d9b8aec11cbf7372be194fe9c438bc383e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-26T06:39:34.228 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:33 smithi167 podman[74147]: 2024-02-26 06:39:33.839171184 +0000 UTC m=+0.017716009 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:39:34.228 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:33 smithi167 podman[74147]: 2024-02-26 06:39:33.96852201 +0000 UTC m=+0.147066885 container create 60ac223593c1ae440dd58b97ee62738e90781ea5498caca4fc3113c6b1543c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2) 2024-02-26T06:39:34.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:34 smithi167 podman[74147]: 2024-02-26 06:39:34.229321149 +0000 UTC m=+0.407865967 container init 60ac223593c1ae440dd58b97ee62738e90781ea5498caca4fc3113c6b1543c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-26T06:39:34.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:34 smithi167 podman[74147]: 2024-02-26 06:39:34.232675527 +0000 UTC m=+0.411220346 container start 60ac223593c1ae440dd58b97ee62738e90781ea5498caca4fc3113c6b1543c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5) 2024-02-26T06:39:34.495 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:34 smithi167 podman[74147]: 2024-02-26 06:39:34.337713374 +0000 UTC m=+0.516258215 container attach 60ac223593c1ae440dd58b97ee62738e90781ea5498caca4fc3113c6b1543c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-26T06:39:34.815 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:34 smithi167 podman[74147]: 2024-02-26 06:39:34.5312468 +0000 UTC m=+0.709791616 container died 60ac223593c1ae440dd58b97ee62738e90781ea5498caca4fc3113c6b1543c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-26T06:39:35.074 INFO:journalctl@ceph.osd.2.smithi167.stdout:Feb 26 06:39:35 smithi167 podman[74173]: 2024-02-26 06:39:35.058098549 +0000 UTC m=+0.521847676 container remove 60ac223593c1ae440dd58b97ee62738e90781ea5498caca4fc3113c6b1543c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T06:39:35.079 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.2.service' 2024-02-26T06:39:35.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:35.121 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-26T06:39:35.122 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-26T06:39:35.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.3 2024-02-26T06:39:35.672 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:35 smithi006 systemd[1]: Stopping Ceph osd.3 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:36.172 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:35 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3[53286]: 2024-02-26T06:39:35.725+0000 7f9e68e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:36.172 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:35 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3[53286]: 2024-02-26T06:39:35.725+0000 7f9e68e00700 -1 osd.3 99 *** Got signal Terminated *** 2024-02-26T06:39:36.173 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:35 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3[53286]: 2024-02-26T06:39:35.725+0000 7f9e68e00700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:39:38.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:38.040+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:38.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:38.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:39.037 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:38 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:38.734+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:39.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:39 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:39.037+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:39.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:39.046+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:39.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:39.044+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:39.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:39.019+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:40.031 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:39 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:39.773+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:40.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:40 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:40.031+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:40.494 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:40.043+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:40.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:40.022+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:40.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:40.015+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:40.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:40 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:40.298+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:41.034 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:40 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:40.724+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:41.248 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:40.993+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:41.248 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:41.010+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:41.248 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:40.984+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:41.399 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:41 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:41.033+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:41.399 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:41 smithi006 podman[99780]: 2024-02-26 06:39:41.114421273 +0000 UTC m=+5.517188373 container died 5587f0477a57461b90a194fff4807c7b7d5104b9210e340ea6c617d29a024c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-26T06:39:41.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:41 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:41.333+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:41.672 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:41 smithi006 podman[99780]: 2024-02-26 06:39:41.536485552 +0000 UTC m=+5.939252643 container cleanup 5587f0477a57461b90a194fff4807c7b7d5104b9210e340ea6c617d29a024c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True) 2024-02-26T06:39:41.672 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:41 smithi006 bash[99780]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3 2024-02-26T06:39:42.023 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:41 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:41.687+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:42.023 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:41 smithi006 podman[99799]: 2024-02-26 06:39:41.753738332 +0000 UTC m=+0.628499766 container remove 5587f0477a57461b90a194fff4807c7b7d5104b9210e340ea6c617d29a024c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD) 2024-02-26T06:39:42.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:41.992+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:42.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:41.992+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:42.245 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:42.046+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:42.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:42.022+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:42.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:42 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:42.309+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:42.827 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:42 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:42.714+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:42.827 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:42 smithi006 podman[99888]: 2024-02-26 06:39:42.475840856 +0000 UTC m=+0.041451656 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:39:42.828 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:42 smithi006 podman[99888]: 2024-02-26 06:39:42.621559953 +0000 UTC m=+0.187170748 container create 1d26cd53fbe0bfc522610b224fcd93788688e72be803076541d861375b7bd61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-26T06:39:43.171 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:42 smithi006 podman[99888]: 2024-02-26 06:39:42.934017526 +0000 UTC m=+0.499628353 container init 1d26cd53fbe0bfc522610b224fcd93788688e72be803076541d861375b7bd61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5) 2024-02-26T06:39:43.172 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:42 smithi006 podman[99888]: 2024-02-26 06:39:42.946293008 +0000 UTC m=+0.511903838 container start 1d26cd53fbe0bfc522610b224fcd93788688e72be803076541d861375b7bd61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:39:43.172 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:43 smithi006 podman[99888]: 2024-02-26 06:39:43.03264295 +0000 UTC m=+0.598253800 container attach 1d26cd53fbe0bfc522610b224fcd93788688e72be803076541d861375b7bd61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T06:39:43.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:43 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:43.056+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:43.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:42.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:43.245 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:43.036+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:43.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:43.031+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:43.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:43 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:43.302+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:43.671 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:43 smithi006 podman[99888]: 2024-02-26 06:39:43.395868231 +0000 UTC m=+0.961479088 container died 1d26cd53fbe0bfc522610b224fcd93788688e72be803076541d861375b7bd61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:39:44.014 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:43 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:43.758+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:44.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.3.service' 2024-02-26T06:39:44.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:43.945+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:44.245 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:44.074+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:44.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:44.074+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:44.279 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:44 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:44.013+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:44.280 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:44 smithi006 podman[99914]: 2024-02-26 06:39:44.152696918 +0000 UTC m=+0.738096932 container remove 1d26cd53fbe0bfc522610b224fcd93788688e72be803076541d861375b7bd61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-26T06:39:44.280 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:44 smithi006 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-26T06:39:44.280 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:44 smithi006 systemd[1]: Stopped Ceph osd.3 for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:39:44.280 INFO:journalctl@ceph.osd.3.smithi006.stdout:Feb 26 06:39:44 smithi006 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.3.service: Consumed 28.890s CPU time. 2024-02-26T06:39:44.317 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:44.317 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-26T06:39:44.318 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-26T06:39:44.318 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.4 2024-02-26T06:39:44.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:44.316+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:44.615 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:44 smithi090 systemd[1]: Stopping Ceph osd.4 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:45.048 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:44 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:44.713+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:45.115 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:44.616+0000 7f92bf441700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:45.115 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:44.616+0000 7f92bf441700 -1 osd.4 99 *** Got signal Terminated *** 2024-02-26T06:39:45.115 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:44 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:44.616+0000 7f92bf441700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:39:45.244 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:45.104+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:45.245 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:45.121+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:44.966+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:45.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:45 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:45.062+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:45.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:45 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:45.299+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:46.089 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:45 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:45.687+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:46.089 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:45 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:45.687+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:45.974+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:46.245 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:46.086+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:46.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:46.088+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:46.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:46.088+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:46.615 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:46 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:46.367+0000 7f92b822e700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:22.039664+0000 front 2024-02-26T06:39:22.039820+0000 (oldest deadline 2024-02-26T06:39:46.139837+0000) 2024-02-26T06:39:46.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:46 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:46.259+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:46.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:46.663+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:46.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:46 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:46.663+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:47.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:46 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:46.751+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:47.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:47 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:47.117+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:47.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:47.000+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:47.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:47.117+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:47.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:47.072+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:47.615 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:47 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:47.360+0000 7f92b822e700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:22.039664+0000 front 2024-02-26T06:39:22.039820+0000 (oldest deadline 2024-02-26T06:39:46.139837+0000) 2024-02-26T06:39:47.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:47 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:47.263+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:47.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:47 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:47.635+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:47.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:47 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:47.635+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:48.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:47 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:47.725+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:48.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:48 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:48.073+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:48.494 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:48 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:48.087+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:48.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:48 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:48.048+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:48.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:48 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:48.105+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:48.615 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:48 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:48.328+0000 7f92b822e700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:22.039664+0000 front 2024-02-26T06:39:22.039820+0000 (oldest deadline 2024-02-26T06:39:46.139837+0000) 2024-02-26T06:39:48.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:48 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:48.235+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:48.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:48 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:48.656+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:48.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:48 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:48.656+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:49.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:48 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:48.711+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:49.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:49 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:49.029+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:49.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:49 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:49.082+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:49.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:49 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:49.018+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:49.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:49 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:49.044+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:49.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:49 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:49.189+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:49.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:49 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:49.189+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:49.615 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:49 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4[47388]: 2024-02-26T06:39:49.293+0000 7f92b822e700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:22.039664+0000 front 2024-02-26T06:39:22.039820+0000 (oldest deadline 2024-02-26T06:39:46.139837+0000) 2024-02-26T06:39:49.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:49 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:49.638+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:49.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:49 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:49.638+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:49.941 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:49 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:49.689+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:50.210 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:49 smithi090 podman[73739]: 2024-02-26 06:39:49.941182511 +0000 UTC m=+5.426467179 container died f315cd2920fbce44d067131c18e8e0c6c601a5738542aed692319f49f0f2c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-26T06:39:50.429 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:49 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:49.989+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:50.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:50.096+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:50.496 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:50.090+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:50.496 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:50.090+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:50.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:50.012+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:50.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:50.012+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:50.520 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:50 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:50.215+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:50.520 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:50 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:50.215+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:50.521 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:50 smithi090 podman[73739]: 2024-02-26 06:39:50.302105017 +0000 UTC m=+5.787389694 container cleanup f315cd2920fbce44d067131c18e8e0c6c601a5738542aed692319f49f0f2c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=) 2024-02-26T06:39:50.521 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:50 smithi090 bash[73739]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4 2024-02-26T06:39:50.839 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:50 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:50.662+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:50.839 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:50 smithi090 podman[73757]: 2024-02-26 06:39:50.520004977 +0000 UTC m=+0.580140237 container remove f315cd2920fbce44d067131c18e8e0c6c601a5738542aed692319f49f0f2c8c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:39:50.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:50 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:50.590+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:50.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:50 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:50.590+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:51.107 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:50 smithi090 podman[73847]: 2024-02-26 06:39:50.742452372 +0000 UTC m=+0.017864647 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:39:51.108 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:50 smithi090 podman[73847]: 2024-02-26 06:39:50.846188279 +0000 UTC m=+0.121600587 container create e9f342d0815f622ffb4b67a3aae41bd9b3d5e63c005619e403ba7f4aed4240ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T06:39:51.365 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:51 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:51.189+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:51.365 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:51 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:51.189+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:51.366 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 podman[73847]: 2024-02-26 06:39:51.107305961 +0000 UTC m=+0.382718237 container init e9f342d0815f622ffb4b67a3aae41bd9b3d5e63c005619e403ba7f4aed4240ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:39:51.366 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 podman[73847]: 2024-02-26 06:39:51.111061131 +0000 UTC m=+0.386473405 container start e9f342d0815f622ffb4b67a3aae41bd9b3d5e63c005619e403ba7f4aed4240ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:39:51.366 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 podman[73847]: 2024-02-26 06:39:51.190730893 +0000 UTC m=+0.466143173 container attach e9f342d0815f622ffb4b67a3aae41bd9b3d5e63c005619e403ba7f4aed4240ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T06:39:51.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:50.999+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:51.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:50.999+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:51.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:51 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:51.087+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:51.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:51 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:51.132+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:51.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:51 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:51.132+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:51.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:50.997+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:51.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:50 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:50.997+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:51.646 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 podman[73847]: 2024-02-26 06:39:51.394492658 +0000 UTC m=+0.669904919 container died e9f342d0815f622ffb4b67a3aae41bd9b3d5e63c005619e403ba7f4aed4240ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-02-26T06:39:51.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:51.630+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:52.500 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:51.630+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:52.502 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:52.103+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:52.502 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:51 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:51.645+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:52.502 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:51.974+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:52.502 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:51.974+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:52.503 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:52.133+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:52.503 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:52.133+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:52.503 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:52.016+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:52.503 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:52.016+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:52.504 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.4.service' 2024-02-26T06:39:52.505 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 podman[73873]: 2024-02-26 06:39:51.937740581 +0000 UTC m=+0.534445372 container remove e9f342d0815f622ffb4b67a3aae41bd9b3d5e63c005619e403ba7f4aed4240ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:39:52.505 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-26T06:39:52.505 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 systemd[1]: Stopped Ceph osd.4 for 5e92f702-d46e-11ee-95c3-87774f69a715. 2024-02-26T06:39:52.505 INFO:journalctl@ceph.osd.4.smithi090.stdout:Feb 26 06:39:51 smithi090 systemd[1]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.4.service: Consumed 12.447s CPU time. 2024-02-26T06:39:52.524 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:52 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:52.232+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:52.524 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:52 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:52.232+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:52.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:39:52.556 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-26T06:39:52.556 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-26T06:39:52.556 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.5 2024-02-26T06:39:52.779 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:52 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:52.661+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:52.851 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:52 smithi167 systemd[1]: Stopping Ceph osd.5 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:39:52.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:52 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:52.609+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:52.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:52 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:52.609+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:53.152 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:53 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:53.096+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:53.153 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:53 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:53.037+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:53.153 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:53 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:53.037+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:53.153 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:52.851+0000 7f3f85c7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:39:53.153 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:52.851+0000 7f3f85c7a700 -1 osd.5 99 *** Got signal Terminated *** 2024-02-26T06:39:53.153 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:52 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:52.851+0000 7f3f85c7a700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:39:53.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:52 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:52.931+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:53.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:52 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:52.931+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:53.494 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:53 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:53.153+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:53.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:53 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:53.153+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:53.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:53 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:53.263+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:53.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:53 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:53.263+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:53.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:53 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:53.622+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:53.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:53 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:53.622+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:54.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:53 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:53.624+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:54.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:53 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:53.926+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:54.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:53 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:53.926+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:54.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:54 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:54.106+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:54.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:54 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:54.157+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:54.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:54 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:54.157+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:54.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:54 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:54.074+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:54.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:54 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:54.074+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:54.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:54 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:54.217+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:54.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:54 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:54.217+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:54.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:54 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:54.217+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:39:54.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:54 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:54.642+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:54.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:54 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:54.642+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:55.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:54 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:54.647+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:55.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:54 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:54.647+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:39:55.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:54 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:54.947+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:55.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:54 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:54.947+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:55.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:55 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:55.147+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:55.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:55 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:55.043+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:55.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:55 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:55.043+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:55.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:55 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:55.170+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:55.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:55 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:55.170+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:55.496 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:55 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:55.170+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.178643+0000 front 2024-02-26T06:39:29.178609+0000 (oldest deadline 2024-02-26T06:39:54.478797+0000) 2024-02-26T06:39:55.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:55 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:55.193+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:55.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:55 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:55.193+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:55.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:55 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:55.193+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:39:55.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:55 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:55.630+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:55.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:55 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:55.630+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:56.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:55 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:55.693+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:56.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:55 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:55.693+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:39:56.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:55 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:55.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:56.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:55 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:55.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:56.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:56.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:56.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:56.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:56.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:56.133+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:56.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:56.133+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:39:56.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:56.220+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:56.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:56.220+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:56.496 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:56 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:56.220+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.178643+0000 front 2024-02-26T06:39:29.178609+0000 (oldest deadline 2024-02-26T06:39:54.478797+0000) 2024-02-26T06:39:56.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:56.238+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:56.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:56.238+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:56.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:56.238+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:39:56.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:56.612+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:56.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:56.612+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:57.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:56.706+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:57.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:56 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:56.706+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:39:57.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:56.945+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:57.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:56.945+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:57.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:56.945+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:39:57.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:57.166+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:57.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:57.166+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:39:57.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:57.012+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:57.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:57.012+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:57.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:57.012+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:39:57.495 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:57.256+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.176432+0000 front 2024-02-26T06:39:15.176307+0000 (oldest deadline 2024-02-26T06:39:38.676287+0000) 2024-02-26T06:39:57.496 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:57.256+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.878084+0000 front 2024-02-26T06:39:23.878208+0000 (oldest deadline 2024-02-26T06:39:49.178175+0000) 2024-02-26T06:39:57.496 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:57 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5[48263]: 2024-02-26T06:39:57.256+0000 7f3f82a6f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.178643+0000 front 2024-02-26T06:39:29.178609+0000 (oldest deadline 2024-02-26T06:39:54.478797+0000) 2024-02-26T06:39:57.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:57.242+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:57.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:57.242+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:57.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:57.242+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:39:57.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:57.564+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:57.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:57.564+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:57.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:57.564+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:39:58.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:57.699+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:58.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:57 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:57.699+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:39:58.399 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:58.127+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:58.399 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:58.127+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:39:58.399 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:58.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:58.399 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:58.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:58.399 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:58 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:58.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:39:58.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:57.927+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:58.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:57.927+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:58.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:57 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:57.927+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:39:58.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:58.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:58.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:58.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:58.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:58.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:39:58.660 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:58 smithi167 podman[74265]: 2024-02-26 06:39:58.399654262 +0000 UTC m=+5.648368944 container died 6f64f59f7a3fdceb34fce06098e3672c17c3843f1cafb39ae27152e8d71b95e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T06:39:58.919 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:58 smithi167 podman[74265]: 2024-02-26 06:39:58.743547158 +0000 UTC m=+5.992261833 container cleanup 6f64f59f7a3fdceb34fce06098e3672c17c3843f1cafb39ae27152e8d71b95e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-26T06:39:58.919 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:58 smithi167 bash[74265]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5 2024-02-26T06:39:58.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:58.560+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:58.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:58.560+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:58.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:58.560+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:39:59.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:58.653+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:39:59.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:58 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:58.653+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:39:59.244 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:58 smithi167 podman[74282]: 2024-02-26 06:39:58.91999952 +0000 UTC m=+0.521019356 container remove 6f64f59f7a3fdceb34fce06098e3672c17c3843f1cafb39ae27152e8d71b95e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-26T06:39:59.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:59.145+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:39:59.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:39:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:39:59.145+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:39:59.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:59.073+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:39:59.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:59.073+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:39:59.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:39:59 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:39:59.073+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:39:59.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:58.934+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:39:59.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:58.934+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:39:59.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:58 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:58.934+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:39:59.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:59.312+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:39:59.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:59.312+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:39:59.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:39:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:39:59.312+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:39:59.664 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:59 smithi167 podman[74375]: 2024-02-26 06:39:59.345831271 +0000 UTC m=+0.103965923 container create 5ed56dcb4f54ee0237fba0a1a3b5f612dbe4053f4f811b42e18403cbd6b03ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-26T06:39:59.664 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:59 smithi167 podman[74375]: 2024-02-26 06:39:59.260260212 +0000 UTC m=+0.018394889 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:39:59.664 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:59 smithi167 podman[74375]: 2024-02-26 06:39:59.589799588 +0000 UTC m=+0.347934249 container init 5ed56dcb4f54ee0237fba0a1a3b5f612dbe4053f4f811b42e18403cbd6b03ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T06:39:59.664 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:59 smithi167 podman[74375]: 2024-02-26 06:39:59.592970564 +0000 UTC m=+0.351105217 container start 5ed56dcb4f54ee0237fba0a1a3b5f612dbe4053f4f811b42e18403cbd6b03ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-26T06:39:59.664 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:59 smithi167 podman[74375]: 2024-02-26 06:39:59.664811917 +0000 UTC m=+0.422946590 container attach 5ed56dcb4f54ee0237fba0a1a3b5f612dbe4053f4f811b42e18403cbd6b03ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-26T06:39:59.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:59.606+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:39:59.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:59.606+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:39:59.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:39:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:39:59.606+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:39:59.995 INFO:journalctl@ceph.osd.5.smithi167.stdout:Feb 26 06:39:59 smithi167 podman[74375]: 2024-02-26 06:39:59.887082248 +0000 UTC m=+0.645216904 container died 5ed56dcb4f54ee0237fba0a1a3b5f612dbe4053f4f811b42e18403cbd6b03ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-26T06:40:00.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:59.689+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:00.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:39:59 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:39:59.689+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:00.424 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:59.955+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:00.424 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:59.955+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:00.424 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:39:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:39:59.955+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:00.485 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:00.133+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:00.485 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:00.134+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:00.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:00.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:00.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:00.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:00.486 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:00 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:00.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:00.510 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.5.service' 2024-02-26T06:40:00.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:00.551 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-26T06:40:00.551 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-26T06:40:00.552 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.6 2024-02-26T06:40:00.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:00.293+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:00.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:00.293+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:00.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:00.293+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:00.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:00.618+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:40:00.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:00.618+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:40:00.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:00.618+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:40:00.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:00 smithi006 systemd[1]: Stopping Ceph osd.6 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:01.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:00.706+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:01.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:00 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:00.706+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:01.422 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.140+0000 7f44a7000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:01.422 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.140+0000 7f44a7000700 -1 osd.6 99 *** Got signal Terminated *** 2024-02-26T06:40:01.422 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.140+0000 7f44a7000700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:01.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:00.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:01.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:00.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:01.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:00 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:00.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:01.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:01.178+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:01.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:01.178+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:01.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:01.107+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:01.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:01.107+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:01.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:01 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:01.107+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:01.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:01.298+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:01.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:01.298+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:01.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:01.298+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:01.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.657+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:40:01.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.657+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:40:01.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.657+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:40:01.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:01.657+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.236430+0000 front 2024-02-26T06:39:40.236363+0000 (oldest deadline 2024-02-26T06:40:01.335919+0000) 2024-02-26T06:40:02.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:01.707+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:02.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:01 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:01.707+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:02.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:01.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:02.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:01.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:02.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:01.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:02.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:01 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:01.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:02.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:02.206+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:02.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:02.206+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:02.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:02.140+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:02.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:02.140+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:02.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:02 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:02.140+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:02.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:02 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:02.300+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:02.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:02 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:02.300+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:02.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:02 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:02.300+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:02.919 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:02.635+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:40:02.920 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:02.635+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:40:02.920 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:02.635+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:40:02.920 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:02.635+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.236430+0000 front 2024-02-26T06:39:40.236363+0000 (oldest deadline 2024-02-26T06:40:01.335919+0000) 2024-02-26T06:40:03.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:02 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:02.668+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:03.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:02 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:02.668+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:03.171 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:02.918+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:03.171 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:02.918+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:03.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:02.918+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:03.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:02 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:02.918+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:03.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:03.184+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:03.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:03.184+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:03.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:03.184+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:03.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:03.092+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:03.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:03.092+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:03.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:03 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:03.092+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:03.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:03.346+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:03.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:03.346+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:03.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:03.346+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:03.921 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:03.655+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:40:03.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:03.655+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:40:03.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:03.655+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:40:03.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:03.655+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.236430+0000 front 2024-02-26T06:39:40.236363+0000 (oldest deadline 2024-02-26T06:40:01.335919+0000) 2024-02-26T06:40:04.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:03.696+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:04.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:03.696+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:04.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:03 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:03.696+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:04.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:03.951+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:04.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:03.951+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:04.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:03.951+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:04.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:03 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:03.951+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:04.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:04.204+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:04.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:04.204+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:04.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:04.204+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:04.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:04.080+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:04.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:04.080+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:04.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:04 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:04.080+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:04.655 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:04 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:04.380+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:04.655 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:04 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:04.380+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:04.655 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:04 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:04.380+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:04.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:04.646+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:40:04.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:04.646+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:40:04.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:04.646+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:40:04.922 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:04.646+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.236430+0000 front 2024-02-26T06:39:40.236363+0000 (oldest deadline 2024-02-26T06:40:01.335919+0000) 2024-02-26T06:40:05.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:04 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:04.655+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:05.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:04 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:04.655+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:05.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:04 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:04.655+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:05.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:04.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:05.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:04.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:05.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:04.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:05.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:04 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:04.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:05.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:05 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:05.251+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:05.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:05 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:05.251+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:05.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:05 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:05.251+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:05.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:05 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:05.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:05.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:05 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:05.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:05.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:05 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:05.095+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:05.651 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:05.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:05.651 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:05.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:05.652 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:05.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:05.652 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:05.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:06.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:05.651+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:06.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:05.651+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:06.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:05 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:05.651+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:06.172 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:05.677+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:15.827067+0000 front 2024-02-26T06:39:15.827833+0000 (oldest deadline 2024-02-26T06:39:38.127148+0000) 2024-02-26T06:40:06.172 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:05.677+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:24.530790+0000 front 2024-02-26T06:39:24.530867+0000 (oldest deadline 2024-02-26T06:39:45.030845+0000) 2024-02-26T06:40:06.172 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:05.677+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:26.733575+0000 front 2024-02-26T06:39:26.733237+0000 (oldest deadline 2024-02-26T06:39:56.734446+0000) 2024-02-26T06:40:06.172 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6[59850]: 2024-02-26T06:40:05.677+0000 7f449e200700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.236430+0000 front 2024-02-26T06:39:40.236363+0000 (oldest deadline 2024-02-26T06:40:01.335919+0000) 2024-02-26T06:40:06.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:05.925+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:06.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:05.925+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:06.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:05.925+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:06.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:05.925+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:06.396 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:06.243+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:06.396 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:06.243+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:06.396 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:06.243+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:06.396 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:06.056+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:06.397 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:06.056+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:06.397 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:06 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:06.056+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:06.865 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:06.637+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:06.865 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:06.637+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:06.865 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:06.637+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:06.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:06.388+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:06.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:06.388+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:06.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:06.388+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:06.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:06 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:06.388+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:07.035 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:06 smithi006 podman[100007]: 2024-02-26 06:40:06.717122003 +0000 UTC m=+5.738061470 container died b4f960800e7b834de2fe4a23081443a75016d91ae1bed7ca88953604e3ef185c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:07.036 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:06.897+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:07.036 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:06.897+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:07.036 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:06.897+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:07.036 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:06 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:06.897+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:07.406 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:07 smithi006 podman[100007]: 2024-02-26 06:40:07.163875783 +0000 UTC m=+6.184815297 container cleanup b4f960800e7b834de2fe4a23081443a75016d91ae1bed7ca88953604e3ef185c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:07.407 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:07 smithi006 bash[100007]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6 2024-02-26T06:40:07.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:07 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:07.240+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:07.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:07 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:07.240+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:07.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:07 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:07.240+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:07.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:07 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:07.041+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:07.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:07 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:07.041+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:07.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:07 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:07.041+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:07.637 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:07.384+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:07.637 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:07.384+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:07.637 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:07.384+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:07.637 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:07.384+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:07.671 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:07 smithi006 podman[100026]: 2024-02-26 06:40:07.406565068 +0000 UTC m=+0.678974691 container remove b4f960800e7b834de2fe4a23081443a75016d91ae1bed7ca88953604e3ef185c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5) 2024-02-26T06:40:08.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:07.637+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:08.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:07.637+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:08.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:07 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:07.637+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:08.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:07.858+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:08.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:07.858+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:08.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:07.858+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:08.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:07 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:07.858+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:08.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:08 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:08.209+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:08.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:08 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:08.209+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:08.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:08 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:08.209+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:08.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:08 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:08.046+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:08.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:08 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:08.046+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:08.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:08 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:08.046+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:08.553 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:08 smithi006 podman[100115]: 2024-02-26 06:40:08.14003231 +0000 UTC m=+0.038266153 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:08.553 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:08 smithi006 podman[100115]: 2024-02-26 06:40:08.274214562 +0000 UTC m=+0.172448434 container create 6c26bcb04b72b9e7aae38fc0adcbfa9ae29f629072e0dbbed40d0994b35f39f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:08.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:08.364+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:08.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:08.364+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:08.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:08.364+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:08.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:08.364+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:08.909 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:08 smithi006 podman[100115]: 2024-02-26 06:40:08.553174148 +0000 UTC m=+0.451408044 container init 6c26bcb04b72b9e7aae38fc0adcbfa9ae29f629072e0dbbed40d0994b35f39f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:08.909 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:08 smithi006 podman[100115]: 2024-02-26 06:40:08.56516237 +0000 UTC m=+0.463396238 container start 6c26bcb04b72b9e7aae38fc0adcbfa9ae29f629072e0dbbed40d0994b35f39f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T06:40:08.909 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:08 smithi006 podman[100115]: 2024-02-26 06:40:08.70168333 +0000 UTC m=+0.599917245 container attach 6c26bcb04b72b9e7aae38fc0adcbfa9ae29f629072e0dbbed40d0994b35f39f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214) 2024-02-26T06:40:09.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:08.659+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:09.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:08.659+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:09.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:08 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:08.659+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:09.172 INFO:journalctl@ceph.osd.6.smithi006.stdout:Feb 26 06:40:09 smithi006 podman[100115]: 2024-02-26 06:40:09.014557731 +0000 UTC m=+0.912791656 container died 6c26bcb04b72b9e7aae38fc0adcbfa9ae29f629072e0dbbed40d0994b35f39f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-26T06:40:09.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:08.907+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:09.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:08.907+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:09.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:08.907+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:09.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:08 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:08.907+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:09.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:09 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:09.232+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:09.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:09 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:09.232+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:09.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:09 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:09.232+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:09.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:09 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:09.035+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:09.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:09 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:09.035+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:09.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:09 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:09.035+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:09.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:09.333+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:09.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:09.333+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:09.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:09.333+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:09.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:09.333+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:09.751 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.6.service' 2024-02-26T06:40:09.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:09.885 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-26T06:40:09.885 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-26T06:40:09.885 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.7 2024-02-26T06:40:09.941 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:09.689+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:09.942 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:09.689+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:09.942 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:09 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:09.689+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:10.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:09.932+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:10.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:09.932+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:10.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:09.932+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:10.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:09 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:09.932+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:10.290 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:09 smithi090 systemd[1]: Stopping Ceph osd.7 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:10.290 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:10.183+0000 7f434d7ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:10.290 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:10.183+0000 7f434d7ea700 -1 osd.7 99 *** Got signal Terminated *** 2024-02-26T06:40:10.290 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:10.183+0000 7f434d7ea700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:10.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:10 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:10.208+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:10.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:10 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:10.208+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:10.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:10 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:10.208+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:10.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:10 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:10.071+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:10.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:10 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:10.071+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:10.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:10 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:10.071+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:10.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:10.290+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:10.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:10.290+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:10.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:10.290+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:10.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:10.290+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:11.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:10.650+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:11.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:10.650+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:11.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:10 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:10.650+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:11.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:10.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:11.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:10.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:11.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:10.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:11.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:10 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:10.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:11.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:11.217+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:11.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:11.217+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:11.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:11.217+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:11.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:11.114+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:11.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:11.114+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:11.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:11.114+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:11.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:11 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:11.114+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:11.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:11.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:11.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:11.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:11.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:11.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:11.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:11.273+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:12.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:11.700+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:12.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:11.700+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:12.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:11.700+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:12.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:11.952+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:12.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:11.952+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:12.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:11.952+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:12.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:11 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:11.952+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:12.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:12.111+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:12.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:12.111+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:12.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:12.111+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:12.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:12.111+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:12.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:12.186+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:12.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:12.186+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:12.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:12.186+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:12.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:12 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:12.186+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:12.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:12.307+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:12.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:12.307+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:12.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:12.307+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:12.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:12.307+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:13.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:12.728+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:13.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:12.728+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:13.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:12 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:12.728+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:13.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:12 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:12.966+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:13.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:12 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:12.966+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:13.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:12 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:12.966+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:13.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:12 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:12.966+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:13.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:12 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:12.966+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:13.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:13.087+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:13.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:13.087+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:13.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:13.087+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:13.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:13.087+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:13.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:13.202+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:13.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:13.202+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:13.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:13.202+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:13.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:13 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:13.202+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:13.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:13.354+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:13.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:13.354+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:13.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:13.354+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:13.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:13.354+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:13.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:13.354+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:14.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:13.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:14.115 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:13.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:14.116 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:13 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:13.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:14.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:13.938+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:14.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:13.938+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:14.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:13.938+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:14.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:13.938+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:14.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:13 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:13.938+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:14.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:14.052+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:14.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:14.052+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:14.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:14.052+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:14.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:14.052+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:14.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:14.200+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:14.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:14.200+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:14.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:14.200+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:14.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:14 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:14.200+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:14.680 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:14.389+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:14.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:14.389+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:14.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:14.389+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:14.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:14.389+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:14.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:14.389+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:15.116 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:14.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.604900+0000 front 2024-02-26T06:39:23.604851+0000 (oldest deadline 2024-02-26T06:39:46.504993+0000) 2024-02-26T06:40:15.116 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:14.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:29.406215+0000 front 2024-02-26T06:39:29.406181+0000 (oldest deadline 2024-02-26T06:39:54.105982+0000) 2024-02-26T06:40:15.116 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:14.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:36.907765+0000 front 2024-02-26T06:39:36.907631+0000 (oldest deadline 2024-02-26T06:40:02.807526+0000) 2024-02-26T06:40:15.116 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:14 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7[54549]: 2024-02-26T06:40:14.681+0000 7f43465d7700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.608799+0000 front 2024-02-26T06:39:48.608761+0000 (oldest deadline 2024-02-26T06:40:13.908898+0000) 2024-02-26T06:40:15.421 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:14.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:15.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:14.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:15.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:14.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:15.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:14.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:15.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:14 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:14.960+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:15.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:15.193+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:15.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:15.193+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:15.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:15.193+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:15.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:15.193+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:15.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:15.037+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:15.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:15.037+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:15.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:15.037+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:15.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:15 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:15.037+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:15.680 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:15.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:15.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:15.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:15.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:15.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:15.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:15.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:15.681 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:15.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:15.959 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:15 smithi090 podman[73969]: 2024-02-26 06:40:15.68130617 +0000 UTC m=+5.594541521 container died 8247d7822dff2bcd6e3c3d1f0bbdcb9f6ebe41a96bccd77b51f16e5441238e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:16.295 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[73969]: 2024-02-26 06:40:16.036509261 +0000 UTC m=+5.949744593 container cleanup 8247d7822dff2bcd6e3c3d1f0bbdcb9f6ebe41a96bccd77b51f16e5441238e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T06:40:16.296 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 bash[73969]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7 2024-02-26T06:40:16.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:15.956+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:16.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:15.956+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:16.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:15.956+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:16.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:15.956+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:16.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:15 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:15.956+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:16.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:16.207+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:16.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:16.207+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:16.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:16.207+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:16.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:16.207+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:16.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:16.074+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:16.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:16.074+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:16.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:16.074+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:16.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:16 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:16.074+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:16.615 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[73986]: 2024-02-26 06:40:16.296305845 +0000 UTC m=+0.615638458 container remove 8247d7822dff2bcd6e3c3d1f0bbdcb9f6ebe41a96bccd77b51f16e5441238e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-26T06:40:16.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:16 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:16.375+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:16.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:16 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:16.375+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:16.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:16 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:16.375+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:16.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:16 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:16.375+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:16.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:16 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:16.375+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:16.908 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[74079]: 2024-02-26 06:40:16.649162016 +0000 UTC m=+0.113773783 container create 10d7d8789b3d895d232f61daaa1988c3d863614b008688de566225596f8336f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T06:40:16.948 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[74079]: 2024-02-26 06:40:16.552690902 +0000 UTC m=+0.017302675 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:16.952 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[74079]: 2024-02-26 06:40:16.90915195 +0000 UTC m=+0.373763734 container init 10d7d8789b3d895d232f61daaa1988c3d863614b008688de566225596f8336f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:17.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:16 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:16.917+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:17.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:16 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:16.917+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:17.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:16 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:16.917+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:17.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:16 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:16.917+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:17.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:16 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:16.917+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:17.192 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[74079]: 2024-02-26 06:40:16.912480329 +0000 UTC m=+0.377092089 container start 10d7d8789b3d895d232f61daaa1988c3d863614b008688de566225596f8336f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-26T06:40:17.192 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:16 smithi090 podman[74079]: 2024-02-26 06:40:16.99217155 +0000 UTC m=+0.456783347 container attach 10d7d8789b3d895d232f61daaa1988c3d863614b008688de566225596f8336f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T06:40:17.455 INFO:journalctl@ceph.osd.7.smithi090.stdout:Feb 26 06:40:17 smithi090 podman[74079]: 2024-02-26 06:40:17.19355467 +0000 UTC m=+0.658166448 container died 10d7d8789b3d895d232f61daaa1988c3d863614b008688de566225596f8336f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-7-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T06:40:17.456 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:17 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:17.385+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:17.456 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:17 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:17.385+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:17.456 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:17 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:17.385+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:17.456 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:17 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:17.385+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:17.456 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:17 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:17.385+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:17.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:17.054+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:17.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:17.054+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:17.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:17.054+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:17.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:17.054+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:17.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:17.199+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:17.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:17.199+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:17.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:17.199+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:17.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:17 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:17.199+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:17.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.7.service' 2024-02-26T06:40:17.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:17.807 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-26T06:40:17.808 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-26T06:40:17.808 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.8 2024-02-26T06:40:18.121 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:17 smithi167 systemd[1]: Stopping Ceph osd.8 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:18.121 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:18.019+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:18.121 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:18.019+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:18.122 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:18.019+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:18.122 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:18.019+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:18.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:17.933+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:18.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:17.933+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:18.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:17.933+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:18.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:17.933+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:18.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:17 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:17.933+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:18.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.121+0000 7f428a955700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:18.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.121+0000 7f428a955700 -1 osd.8 99 *** Got signal Terminated *** 2024-02-26T06:40:18.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.121+0000 7f428a955700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:18.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.203+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:18.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.203+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:18.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.203+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:18.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:18 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:18.203+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:18.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:18 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:18.350+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:18.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:18 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:18.350+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:18.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:18 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:18.350+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:18.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:18 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:18.350+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:18.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:18 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:18.350+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:19.171 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:18.916+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:19.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:18.916+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:19.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:18.916+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:19.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:18.916+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:19.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:18 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:18.916+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:19.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:19.048+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:19.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:19.048+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:19.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:19.048+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:19.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:19.048+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:19.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:19.229+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:19.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:19.229+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:19.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:19.229+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:19.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:19 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:19.229+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:19.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:19.343+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:19.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:19.343+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:19.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:19.343+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:19.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:19.343+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:19.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:19.343+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:20.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:19.939+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:20.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:19.939+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:20.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:19.939+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:20.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:19.939+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:20.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:19.939+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:20.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:19 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:19.939+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:20.494 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:20.224+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:20.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:20.224+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:20.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:20.224+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:20.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:20.224+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:20.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:20.045+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:20.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:20.045+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:20.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:20.045+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:20.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:20 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:20.045+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:20.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:20 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:20.337+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:20.615 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:20 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:20.337+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:20.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:20 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:20.337+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:20.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:20 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:20.337+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:20.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:20 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:20.337+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:20.616 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:20 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:20.337+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:21.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:20.899+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:21.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:20.899+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:21.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:20.899+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:21.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:20.899+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:21.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:20.899+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:21.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:20 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:20.899+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:21.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:21.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:21.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:21.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:21.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:21.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:21.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:21.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:21.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:21.256+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:21.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:21.256+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:21.496 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:21.256+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:21.497 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:21 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:21.256+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:21.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:21 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:21.377+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:21.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:21 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:21.377+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:21.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:21 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:21.377+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:21.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:21 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:21.377+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:21.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:21 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:21.377+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:21.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:21 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:21.377+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:22.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:21.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:22.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:21.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:22.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:21.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:22.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:21.948+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:22.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:21.949+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:22.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:21 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:21.949+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:22.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:22.300+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.939645+0000 front 2024-02-26T06:39:14.939679+0000 (oldest deadline 2024-02-26T06:39:37.239345+0000) 2024-02-26T06:40:22.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:22.300+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.141632+0000 front 2024-02-26T06:39:30.141664+0000 (oldest deadline 2024-02-26T06:39:56.041835+0000) 2024-02-26T06:40:22.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:22.300+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.543152+0000 front 2024-02-26T06:39:39.542991+0000 (oldest deadline 2024-02-26T06:40:03.042914+0000) 2024-02-26T06:40:22.495 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8[55302]: 2024-02-26T06:40:22.300+0000 7f4283742700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:46.543966+0000 front 2024-02-26T06:39:46.543868+0000 (oldest deadline 2024-02-26T06:40:11.843895+0000) 2024-02-26T06:40:22.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:22.078+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:22.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:22.078+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:22.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:22.078+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:22.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:22 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:22.078+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:22.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:22.406+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:22.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:22.406+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:22.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:22.406+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:22.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:22.406+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:22.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:22.406+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:22.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:22 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:22.406+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:23.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:22 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:22.995+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:23.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:22 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:22.995+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:23.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:22 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:22.995+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:23.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:22 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:22.995+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:23.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:22 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:22.995+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:23.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:22 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:22.995+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:23.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:23 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:23.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:23.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:23 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:23.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:23.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:23 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:23.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:23.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:23 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:23.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:23.496 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:23 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:23.063+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:23.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:23.436+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:23.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:23.436+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:23.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:23.436+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:23.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:23.436+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:23.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:23.436+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:23.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:23.436+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:23.879 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:23 smithi167 podman[74482]: 2024-02-26 06:40:23.555002501 +0000 UTC m=+5.547799092 container died 4d65b9a62d2cc6c1b3aee6d31feb1346d65f52425df1408aa0ee6679a01e3bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:40:24.244 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:23 smithi167 podman[74482]: 2024-02-26 06:40:23.880049064 +0000 UTC m=+5.872845653 container cleanup 4d65b9a62d2cc6c1b3aee6d31feb1346d65f52425df1408aa0ee6679a01e3bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-26T06:40:24.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:23 smithi167 bash[74482]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8 2024-02-26T06:40:24.245 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:24 smithi167 podman[74499]: 2024-02-26 06:40:24.073227495 +0000 UTC m=+0.516713057 container remove 4d65b9a62d2cc6c1b3aee6d31feb1346d65f52425df1408aa0ee6679a01e3bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:24.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:24 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:24.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:24.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:24 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:24.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:24.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:24 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:24.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:24.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:24 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:24.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:24.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:24 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:24.049+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:24.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:23.977+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:24.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:23.977+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:24.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:23.977+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:24.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:23.977+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:24.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:23.977+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:24.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:23 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:23.977+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:24.744 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:24 smithi167 podman[74590]: 2024-02-26 06:40:24.395492197 +0000 UTC m=+0.018025614 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:24.745 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:24 smithi167 podman[74590]: 2024-02-26 06:40:24.492349497 +0000 UTC m=+0.114882924 container create 0ac98a1ec0b685c77b000220b73239ae87cf7161337acc161d857d510e057a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T06:40:24.745 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:24 smithi167 podman[74590]: 2024-02-26 06:40:24.703766074 +0000 UTC m=+0.326299495 container init 0ac98a1ec0b685c77b000220b73239ae87cf7161337acc161d857d510e057a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-26T06:40:24.745 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:24 smithi167 podman[74590]: 2024-02-26 06:40:24.707000292 +0000 UTC m=+0.329533700 container start 0ac98a1ec0b685c77b000220b73239ae87cf7161337acc161d857d510e057a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T06:40:24.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:24.409+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:24.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:24.409+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:24.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:24.409+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:24.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:24.409+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:24.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:24.409+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:24.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:24 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:24.409+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:24.999 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:24 smithi167 podman[74590]: 2024-02-26 06:40:24.778639898 +0000 UTC m=+0.401173311 container attach 0ac98a1ec0b685c77b000220b73239ae87cf7161337acc161d857d510e057a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-26T06:40:25.297 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:25 smithi167 podman[74590]: 2024-02-26 06:40:25.000416316 +0000 UTC m=+0.622949739 container died 0ac98a1ec0b685c77b000220b73239ae87cf7161337acc161d857d510e057a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T06:40:25.298 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:25 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:25.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:25.298 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:25 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:25.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:25.298 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:25 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:25.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:25.298 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:25 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:25.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:25.298 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:25 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:25.055+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:25.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.014+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:25.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.014+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:25.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.014+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:25.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.014+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:25.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.014+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:25.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.014+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:25.557 INFO:journalctl@ceph.osd.8.smithi167.stdout:Feb 26 06:40:25 smithi167 podman[74617]: 2024-02-26 06:40:25.540621943 +0000 UTC m=+0.534971242 container remove 0ac98a1ec0b685c77b000220b73239ae87cf7161337acc161d857d510e057a9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-02-26T06:40:25.562 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.8.service' 2024-02-26T06:40:25.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:25.604 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-26T06:40:25.604 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-26T06:40:25.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.9 2024-02-26T06:40:25.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:25 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:25.459+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:25.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:25 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:25.459+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:25.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:25 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:25.459+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:25.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:25 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:25.459+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:25.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:25 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:25.459+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:25.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:25 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:25.459+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:25.992 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 systemd[1]: Stopping Ceph osd.9 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:26.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.992+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:26.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.992+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.992+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.992+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.992+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:25 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:25.992+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:26 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:26.206+0000 7ff11a400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:26 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:26.206+0000 7ff11a400700 -1 osd.9 99 *** Got signal Terminated *** 2024-02-26T06:40:26.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:26 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:26.206+0000 7ff11a400700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:26.494 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.013+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:26.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.013+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:26.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.013+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:26.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.013+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:26.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.013+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:26.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:26 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:26.469+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:26.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:26 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:26.469+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:26.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:26 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:26.469+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:26.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:26 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:26.469+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:26.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:26 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:26.469+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:26.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:26 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:26.469+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:27.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.990+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:27.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.990+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:27.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.990+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:27.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.990+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:27.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.990+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:27.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:26 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:26.990+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:27.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.038+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:27.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.038+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:27.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.038+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:27.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.038+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:27.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.038+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:27.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.038+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:27.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:27.472+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:27.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:27.472+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:27.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:27.472+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:27.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:27.472+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:27.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:27.472+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:27.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:27 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:27.472+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:28.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:27.970+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:28.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:27.970+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:28.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:27.970+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:28.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:27.970+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:28.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:27.970+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:28.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:27.970+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:28.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:28.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:28.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:28.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:28.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:28.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:27 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:27.994+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:28.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:28.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:28.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:28.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:28.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:28.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:28.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:28 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:28.518+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:29.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:28 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:28.984+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:29.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:28 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:28.984+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:29.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:28 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:28.984+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:29.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:28 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:28.984+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:29.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:28 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:28.984+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:29.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:28 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:28.984+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:29.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:29.026+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:29.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:29.026+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:29.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:29.026+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:29.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:29.026+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:29.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:29.026+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:29.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:29 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:29.026+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:29.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:29.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:29.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:29.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:29.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:29.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:29.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:29 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:29.542+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:30.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:30.032+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:30.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:30.032+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:30.172 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:30.032+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:30.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:30.032+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:30.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:30.032+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:30.173 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:30 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:30.032+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:30.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:29 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:29.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:30.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:29 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:29.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:30.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:29 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:29.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:30.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:29 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:29.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:30.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:29 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:29.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:30.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:29 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:29.943+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:30.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:30.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:30.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:30.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:30.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:30.867 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:30.867 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:30 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:30.493+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:31.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:30 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:30.932+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:31.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:30 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:30.932+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:31.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:30 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:30.932+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:31.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:30 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:30.932+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:31.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:30 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:30.932+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:31.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:30 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:30.932+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:31.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:31.034+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.926648+0000 front 2024-02-26T06:39:14.926356+0000 (oldest deadline 2024-02-26T06:39:38.426184+0000) 2024-02-26T06:40:31.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:31.034+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.127979+0000 front 2024-02-26T06:39:23.128057+0000 (oldest deadline 2024-02-26T06:39:50.729707+0000) 2024-02-26T06:40:31.422 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:31.034+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:32.332877+0000 front 2024-02-26T06:39:32.333210+0000 (oldest deadline 2024-02-26T06:39:56.433105+0000) 2024-02-26T06:40:31.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:31.034+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:40.535714+0000 front 2024-02-26T06:39:40.536172+0000 (oldest deadline 2024-02-26T06:40:01.635384+0000) 2024-02-26T06:40:31.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:31.034+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:47.739913+0000 front 2024-02-26T06:39:47.740013+0000 (oldest deadline 2024-02-26T06:40:12.439774+0000) 2024-02-26T06:40:31.423 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9[66531]: 2024-02-26T06:40:31.034+0000 7ff116600700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:55.743385+0000 front 2024-02-26T06:39:55.743305+0000 (oldest deadline 2024-02-26T06:40:19.843387+0000) 2024-02-26T06:40:31.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:31.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:31.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:31.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:31.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:31.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:31.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:31 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:31.460+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:32.043 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:31 smithi006 podman[100230]: 2024-02-26 06:40:31.783508415 +0000 UTC m=+5.709458371 container died d8ded68e1c37afb248e86b00fc10b87a948f3b9ff22bc0df1aab7cf38109b91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:32.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:31.906+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:32.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:31.906+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:32.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:31.906+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:32.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:31.906+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:32.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:31.906+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:32.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:31.906+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:32.341 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:32 smithi006 podman[100230]: 2024-02-26 06:40:32.14050993 +0000 UTC m=+6.066459926 container cleanup d8ded68e1c37afb248e86b00fc10b87a948f3b9ff22bc0df1aab7cf38109b91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:40:32.341 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:32 smithi006 bash[100230]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9 2024-02-26T06:40:32.671 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:32 smithi006 podman[100248]: 2024-02-26 06:40:32.341281119 +0000 UTC m=+0.547247590 container remove d8ded68e1c37afb248e86b00fc10b87a948f3b9ff22bc0df1aab7cf38109b91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T06:40:32.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:32.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:32.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:32.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:32.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:32.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:32.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:32 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:32.433+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:33.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:32 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:32.876+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:33.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:32 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:32.876+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:33.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:32 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:32.876+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:33.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:32 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:32.876+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:33.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:32 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:32.876+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:33.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:32 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:32.876+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:33.337 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:33 smithi006 podman[100334]: 2024-02-26 06:40:33.075540554 +0000 UTC m=+0.038602173 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:33.337 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:33 smithi006 podman[100334]: 2024-02-26 06:40:33.184109193 +0000 UTC m=+0.147170852 container create 7fbeb26bbd48355077b3b9e331e9a81804139397c3f997fe19c8fb1e1bf84944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-26T06:40:33.672 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:33 smithi006 podman[100334]: 2024-02-26 06:40:33.49632751 +0000 UTC m=+0.459389183 container init 7fbeb26bbd48355077b3b9e331e9a81804139397c3f997fe19c8fb1e1bf84944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T06:40:33.672 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:33 smithi006 podman[100334]: 2024-02-26 06:40:33.506480818 +0000 UTC m=+0.469542504 container start 7fbeb26bbd48355077b3b9e331e9a81804139397c3f997fe19c8fb1e1bf84944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:33.672 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:33 smithi006 podman[100334]: 2024-02-26 06:40:33.586650935 +0000 UTC m=+0.549712598 container attach 7fbeb26bbd48355077b3b9e331e9a81804139397c3f997fe19c8fb1e1bf84944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-26T06:40:33.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:33.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:33.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:33.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:33.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:33.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:33.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:33 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:33.441+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:34.171 INFO:journalctl@ceph.osd.9.smithi006.stdout:Feb 26 06:40:33 smithi006 podman[100334]: 2024-02-26 06:40:33.917990781 +0000 UTC m=+0.881052487 container died 7fbeb26bbd48355077b3b9e331e9a81804139397c3f997fe19c8fb1e1bf84944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-9-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-26T06:40:34.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:33 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:33.898+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:34.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:33 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:33.898+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:34.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:33 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:33.898+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:34.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:33 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:33.898+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:34.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:33 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:33.898+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:34.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:33 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:33.898+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:34.670 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.9.service' 2024-02-26T06:40:34.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:34.780 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-26T06:40:34.781 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-26T06:40:34.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.10 2024-02-26T06:40:34.810 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:34.810 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:34.810 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:34.810 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:34.810 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:34.810 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:34.811 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:34.456+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:35.060 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:34 smithi090 systemd[1]: Stopping Ceph osd.10 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:35.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:34.880+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:35.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:34.880+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:35.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:34.880+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:35.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:34.880+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:35.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:34.880+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:35.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:34 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:34.880+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:35.365 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.060+0000 7ffbca3ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:35.365 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.060+0000 7ffbca3ad700 -1 osd.10 99 *** Got signal Terminated *** 2024-02-26T06:40:35.365 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.060+0000 7ffbca3ad700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:35.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:35.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:35.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:35.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:35.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:35.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:35.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:35 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:35.451+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:36.146 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:35.871+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:36.147 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:35.871+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:36.147 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:35.871+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:36.147 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:35.871+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:36.147 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:35.871+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:36.147 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:35.871+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:36.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:36.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:36.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:36.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:36.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:36.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:36.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:36 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:36.428+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:37.160 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:36.910+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:37.160 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:36.910+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:37.161 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:36.910+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:37.161 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:36.910+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:37.161 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:36.910+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:37.161 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:36 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:36.910+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:37.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:37.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:37.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:37.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:37.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:37.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:37.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:37 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:37.405+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:38.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:38.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:38.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:38.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:38.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:38.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:38.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:37 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:37.861+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:38.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:38.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:38.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:38.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:38.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:38.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:38.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:38 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:38.417+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:39.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:39.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:39.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:39.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:39.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:39.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:39.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:38 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:38.853+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:39.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:14.528178+0000 front 2024-02-26T06:39:14.528121+0000 (oldest deadline 2024-02-26T06:39:39.827774+0000) 2024-02-26T06:40:39.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.328624+0000 front 2024-02-26T06:39:23.328728+0000 (oldest deadline 2024-02-26T06:39:48.628829+0000) 2024-02-26T06:40:39.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:28.629624+0000 front 2024-02-26T06:39:28.629541+0000 (oldest deadline 2024-02-26T06:39:53.929440+0000) 2024-02-26T06:40:39.865 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-02-26T06:39:39.731308+0000 front 2024-02-26T06:39:39.731378+0000 (oldest deadline 2024-02-26T06:40:05.030973+0000) 2024-02-26T06:40:39.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:48.432339+0000 front 2024-02-26T06:39:48.432366+0000 (oldest deadline 2024-02-26T06:40:12.532554+0000) 2024-02-26T06:40:39.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:54.233691+0000 front 2024-02-26T06:39:54.233732+0000 (oldest deadline 2024-02-26T06:40:20.133615+0000) 2024-02-26T06:40:39.866 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:39 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10[61647]: 2024-02-26T06:40:39.371+0000 7ffbc71a2700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:05.335637+0000 front 2024-02-26T06:40:05.335707+0000 (oldest deadline 2024-02-26T06:40:28.235244+0000) 2024-02-26T06:40:40.248 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:40.248 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:40.249 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:40.249 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:40.249 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:40.249 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:40.249 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:39.885+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:40.802 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:40 smithi090 podman[74196]: 2024-02-26 06:40:40.470537595 +0000 UTC m=+5.497373775 container died ea2a6e76e2f9c3d049bb65061ff374643b90d91460c0c6d381c057b198a65be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:40:40.802 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:40 smithi090 podman[74196]: 2024-02-26 06:40:40.802483956 +0000 UTC m=+5.829320133 container cleanup ea2a6e76e2f9c3d049bb65061ff374643b90d91460c0c6d381c057b198a65be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-26T06:40:41.116 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:40 smithi090 bash[74196]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10 2024-02-26T06:40:41.116 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74213]: 2024-02-26 06:40:41.00515216 +0000 UTC m=+0.529898845 container remove ea2a6e76e2f9c3d049bb65061ff374643b90d91460c0c6d381c057b198a65be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-26T06:40:41.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:41.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:41.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:41.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:41.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:41.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:41.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:40 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:40.854+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:41.674 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74306]: 2024-02-26 06:40:41.397834721 +0000 UTC m=+0.106301007 container create 0ae201c8d5a21d3cd2bc578cade5e54c94df2d2e875d919c1ffe2a1ea6d50aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:41.675 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74306]: 2024-02-26 06:40:41.309124859 +0000 UTC m=+0.017591153 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:41.675 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74306]: 2024-02-26 06:40:41.600811242 +0000 UTC m=+0.309277539 container init 0ae201c8d5a21d3cd2bc578cade5e54c94df2d2e875d919c1ffe2a1ea6d50aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-02-26T06:40:41.675 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74306]: 2024-02-26 06:40:41.604455885 +0000 UTC m=+0.312922172 container start 0ae201c8d5a21d3cd2bc578cade5e54c94df2d2e875d919c1ffe2a1ea6d50aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T06:40:41.675 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74306]: 2024-02-26 06:40:41.675380481 +0000 UTC m=+0.383846775 container attach 0ae201c8d5a21d3cd2bc578cade5e54c94df2d2e875d919c1ffe2a1ea6d50aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:42.115 INFO:journalctl@ceph.osd.10.smithi090.stdout:Feb 26 06:40:41 smithi090 podman[74306]: 2024-02-26 06:40:41.893119145 +0000 UTC m=+0.601585443 container died 0ae201c8d5a21d3cd2bc578cade5e54c94df2d2e875d919c1ffe2a1ea6d50aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-26T06:40:42.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:42.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:42.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:42.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:42.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:42.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:42.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:41 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:41.887+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:42.505 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.10.service' 2024-02-26T06:40:42.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:42.587 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-26T06:40:42.587 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-26T06:40:42.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.11 2024-02-26T06:40:42.909 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 systemd[1]: Stopping Ceph osd.11 for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:42.909 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:42.909 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:42.910 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:42.910 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:42.910 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:42.910 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:42.910 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.860+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:43.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.909+0000 7fd7b1146700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:43.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.909+0000 7fd7b1146700 -1 osd.11 99 *** Got signal Terminated *** 2024-02-26T06:40:43.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:42 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:42.909+0000 7fd7b1146700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T06:40:44.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:44.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:44.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:44.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:44.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:44.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:44.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:43 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:43.824+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:45.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:45.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:44 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:44.843+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:46.244 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.800+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:46.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:45 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:45.801+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-02-26T06:40:22.460975+0000 front 2024-02-26T06:40:22.460997+0000 (oldest deadline 2024-02-26T06:40:45.360947+0000) 2024-02-26T06:40:47.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:47.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:47.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:47.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:47.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:47.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:47.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:47.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:46 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:46.753+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-02-26T06:40:22.460975+0000 front 2024-02-26T06:40:22.460997+0000 (oldest deadline 2024-02-26T06:40:45.360947+0000) 2024-02-26T06:40:48.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-02-26T06:39:11.250710+0000 front 2024-02-26T06:39:11.250776+0000 (oldest deadline 2024-02-26T06:39:37.150546+0000) 2024-02-26T06:40:48.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-02-26T06:39:23.953647+0000 front 2024-02-26T06:39:23.953430+0000 (oldest deadline 2024-02-26T06:39:49.853565+0000) 2024-02-26T06:40:48.245 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-02-26T06:39:30.954539+0000 front 2024-02-26T06:39:30.954516+0000 (oldest deadline 2024-02-26T06:39:56.254462+0000) 2024-02-26T06:40:48.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-02-26T06:39:45.556689+0000 front 2024-02-26T06:39:45.556727+0000 (oldest deadline 2024-02-26T06:40:10.856795+0000) 2024-02-26T06:40:48.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-02-26T06:39:56.757794+0000 front 2024-02-26T06:39:56.757693+0000 (oldest deadline 2024-02-26T06:40:22.657830+0000) 2024-02-26T06:40:48.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-02-26T06:40:02.658694+0000 front 2024-02-26T06:40:02.658903+0000 (oldest deadline 2024-02-26T06:40:26.758331+0000) 2024-02-26T06:40:48.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-02-26T06:40:13.159998+0000 front 2024-02-26T06:40:13.159924+0000 (oldest deadline 2024-02-26T06:40:37.859742+0000) 2024-02-26T06:40:48.246 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:47 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11[62115]: 2024-02-26T06:40:47.782+0000 7fd7a9f33700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-02-26T06:40:22.460975+0000 front 2024-02-26T06:40:22.460997+0000 (oldest deadline 2024-02-26T06:40:45.360947+0000) 2024-02-26T06:40:48.700 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:48 smithi167 podman[74707]: 2024-02-26 06:40:48.384811132 +0000 UTC m=+5.604571584 container died a2d1f78c06ec4983006926883a5cc1c721dc98c428e07f944cd5d9bac9efa294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:48.700 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:48 smithi167 podman[74707]: 2024-02-26 06:40:48.70099724 +0000 UTC m=+5.920757689 container cleanup a2d1f78c06ec4983006926883a5cc1c721dc98c428e07f944cd5d9bac9efa294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-26T06:40:48.995 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:48 smithi167 bash[74707]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11 2024-02-26T06:40:48.995 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:48 smithi167 podman[74725]: 2024-02-26 06:40:48.893799364 +0000 UTC m=+0.502480443 container remove a2d1f78c06ec4983006926883a5cc1c721dc98c428e07f944cd5d9bac9efa294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-26T06:40:49.616 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:49 smithi167 podman[74816]: 2024-02-26 06:40:49.211898594 +0000 UTC m=+0.017609988 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T06:40:49.616 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:49 smithi167 podman[74816]: 2024-02-26 06:40:49.311177319 +0000 UTC m=+0.116888695 container create 0d9c7482ea7e1c587674c8198a1850e421f71eaa90eec69bbb3298aad337113d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-26T06:40:49.616 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:49 smithi167 podman[74816]: 2024-02-26 06:40:49.533411974 +0000 UTC m=+0.339123363 container init 0d9c7482ea7e1c587674c8198a1850e421f71eaa90eec69bbb3298aad337113d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-26T06:40:49.616 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:49 smithi167 podman[74816]: 2024-02-26 06:40:49.536593474 +0000 UTC m=+0.342304859 container start 0d9c7482ea7e1c587674c8198a1850e421f71eaa90eec69bbb3298aad337113d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:49.617 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:49 smithi167 podman[74816]: 2024-02-26 06:40:49.616600624 +0000 UTC m=+0.422312029 container attach 0d9c7482ea7e1c587674c8198a1850e421f71eaa90eec69bbb3298aad337113d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-26T06:40:49.994 INFO:journalctl@ceph.osd.11.smithi167.stdout:Feb 26 06:40:49 smithi167 podman[74816]: 2024-02-26 06:40:49.830462463 +0000 UTC m=+0.636173868 container died 0d9c7482ea7e1c587674c8198a1850e421f71eaa90eec69bbb3298aad337113d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T06:40:50.435 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@osd.11.service' 2024-02-26T06:40:50.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:50.478 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-26T06:40:50.478 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-26T06:40:50.479 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.a 2024-02-26T06:40:50.922 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:50 smithi006 systemd[1]: Stopping Ceph mds.a for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:51.422 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a[73477]: 2024-02-26T06:40:51.048+0000 7ff365a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:51.422 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:51 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a[73477]: 2024-02-26T06:40:51.048+0000 7ff365a00700 -1 mds.a *** got signal Terminated *** 2024-02-26T06:40:54.422 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:54 smithi006 podman[100449]: 2024-02-26 06:40:54.012691549 +0000 UTC m=+3.097766107 container died 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True) 2024-02-26T06:40:54.835 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:54 smithi006 podman[100449]: 2024-02-26 06:40:54.533184428 +0000 UTC m=+3.618259007 container cleanup 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:40:54.835 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:54 smithi006 bash[100449]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a 2024-02-26T06:40:55.171 INFO:journalctl@ceph.mds.a.smithi006.stdout:Feb 26 06:40:54 smithi006 podman[100475]: 2024-02-26 06:40:54.83454872 +0000 UTC m=+0.808340879 container remove 1fe397810ee45d2176b1a68ae5eeb79431c734c1855d1edf6ef3b4d6adb21b5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-a, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-26T06:40:55.366 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.a.service' 2024-02-26T06:40:55.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:55.491 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-26T06:40:55.492 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-26T06:40:55.492 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.d 2024-02-26T06:40:55.921 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:55 smithi006 systemd[1]: Stopping Ceph mds.d for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:40:56.422 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d[72525]: 2024-02-26T06:40:56.068+0000 7fdff7000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:40:56.422 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:56 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d[72525]: 2024-02-26T06:40:56.068+0000 7fdff7000700 -1 mds.d *** got signal Terminated *** 2024-02-26T06:40:58.326 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:58 smithi006 podman[100579]: 2024-02-26 06:40:58.005660126 +0000 UTC m=+2.073140766 container died 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-26T06:40:58.612 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:58 smithi006 podman[100579]: 2024-02-26 06:40:58.436927941 +0000 UTC m=+2.504408617 container cleanup 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-26T06:40:58.613 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:58 smithi006 bash[100579]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d 2024-02-26T06:40:58.921 INFO:journalctl@ceph.mds.d.smithi006.stdout:Feb 26 06:40:58 smithi006 podman[100593]: 2024-02-26 06:40:58.612676037 +0000 UTC m=+0.586567849 container remove 16c20d0184524f4a070b1b8f1a0bae51e378a7c93420e7a64f6173955941f637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-26T06:40:59.146 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.d.service' 2024-02-26T06:40:59.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:40:59.272 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-26T06:40:59.273 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-26T06:40:59.273 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.g 2024-02-26T06:40:59.672 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:40:59 smithi006 systemd[1]: Stopping Ceph mds.g for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:00.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:40:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g[73136]: 2024-02-26T06:40:59.864+0000 7fc278c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:00.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:40:59 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g[73136]: 2024-02-26T06:40:59.864+0000 7fc278c00700 -1 mds.g *** got signal Terminated *** 2024-02-26T06:41:04.172 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:41:03 smithi006 podman[100696]: 2024-02-26 06:41:03.890152611 +0000 UTC m=+4.178138205 container died e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-26T06:41:04.558 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:41:04 smithi006 podman[100696]: 2024-02-26 06:41:04.340490379 +0000 UTC m=+4.628475950 container cleanup e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:41:04.558 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:41:04 smithi006 bash[100696]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g 2024-02-26T06:41:04.922 INFO:journalctl@ceph.mds.g.smithi006.stdout:Feb 26 06:41:04 smithi006 podman[100710]: 2024-02-26 06:41:04.558397584 +0000 UTC m=+0.658012103 container remove e1e5e0342d405c23996fc34d56623fab66af6dd5a85de05b4ae57dfb99d173a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:05.043 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.g.service' 2024-02-26T06:41:05.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:05.167 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-26T06:41:05.168 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-26T06:41:05.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.j 2024-02-26T06:41:05.672 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:05 smithi006 systemd[1]: Stopping Ceph mds.j for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:06.172 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-j[72160]: 2024-02-26T06:41:05.800+0000 7f1450200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:06.172 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:05 smithi006 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-j[72160]: 2024-02-26T06:41:05.800+0000 7f1450200700 -1 mds.j *** got signal Terminated *** 2024-02-26T06:41:10.107 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:09 smithi006 podman[100816]: 2024-02-26 06:41:09.81613094 +0000 UTC m=+4.203922663 container died be5dd0fd62924eef11d386b097685dc25264ef38c7b1342dc29469a4aee42549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T06:41:10.422 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:10 smithi006 podman[100816]: 2024-02-26 06:41:10.2191111 +0000 UTC m=+4.606902823 container cleanup be5dd0fd62924eef11d386b097685dc25264ef38c7b1342dc29469a4aee42549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:10.422 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:10 smithi006 bash[100816]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-j 2024-02-26T06:41:10.765 INFO:journalctl@ceph.mds.j.smithi006.stdout:Feb 26 06:41:10 smithi006 podman[100842]: 2024-02-26 06:41:10.511753853 +0000 UTC m=+0.678451393 container remove be5dd0fd62924eef11d386b097685dc25264ef38c7b1342dc29469a4aee42549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T06:41:10.783 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.j.service' 2024-02-26T06:41:10.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:10.910 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-26T06:41:10.911 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-26T06:41:10.911 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.b 2024-02-26T06:41:11.193 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:41:10 smithi090 systemd[1]: Stopping Ceph mds.b for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:11.448 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:41:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: 2024-02-26T06:41:11.192+0000 7feff5f32700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:11.449 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:41:11 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b[67335]: 2024-02-26T06:41:11.192+0000 7feff5f32700 -1 mds.b *** got signal Terminated *** 2024-02-26T06:41:14.235 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:41:13 smithi090 podman[74411]: 2024-02-26 06:41:13.951404922 +0000 UTC m=+2.853693731 container died 9a03e2fa1b16d03184bbb396797b6f489931661a9e1d1d51e07909224d97e155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:14.576 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:41:14 smithi090 podman[74411]: 2024-02-26 06:41:14.334638927 +0000 UTC m=+3.236927755 container cleanup 9a03e2fa1b16d03184bbb396797b6f489931661a9e1d1d51e07909224d97e155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:41:14.577 INFO:journalctl@ceph.mds.b.smithi090.stdout:Feb 26 06:41:14 smithi090 bash[74411]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-b 2024-02-26T06:41:14.707 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.b.service' 2024-02-26T06:41:14.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:14.745 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-26T06:41:14.745 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-26T06:41:14.745 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.e 2024-02-26T06:41:15.115 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:14 smithi090 systemd[1]: Stopping Ceph mds.e for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:15.115 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e[66711]: 2024-02-26T06:41:15.062+0000 7fd8defdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:15.116 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:15 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e[66711]: 2024-02-26T06:41:15.062+0000 7fd8defdd700 -1 mds.e *** got signal Terminated *** 2024-02-26T06:41:18.273 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:17 smithi090 podman[74546]: 2024-02-26 06:41:17.95058105 +0000 UTC m=+2.980839972 container died 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:41:18.273 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:18 smithi090 podman[74546]: 2024-02-26 06:41:18.2736373 +0000 UTC m=+3.303896235 container cleanup 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:18.615 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:18 smithi090 bash[74546]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e 2024-02-26T06:41:18.615 INFO:journalctl@ceph.mds.e.smithi090.stdout:Feb 26 06:41:18 smithi090 podman[74559]: 2024-02-26 06:41:18.474997259 +0000 UTC m=+0.518166379 container remove 0eea3ad0d4b96dbb05365774888a27af7c89cb33dc38dc208ee92356dc8f2073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-e, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-26T06:41:18.693 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.e.service' 2024-02-26T06:41:18.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:18.729 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-26T06:41:18.729 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-26T06:41:18.729 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.h 2024-02-26T06:41:19.046 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:41:18 smithi090 systemd[1]: Stopping Ceph mds.h for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:19.366 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:41:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h[67019]: 2024-02-26T06:41:19.043+0000 7ff7483c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:19.366 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:41:19 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h[67019]: 2024-02-26T06:41:19.043+0000 7ff7483c1700 -1 mds.h *** got signal Terminated *** 2024-02-26T06:41:22.228 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:41:21 smithi090 podman[74669]: 2024-02-26 06:41:21.955360245 +0000 UTC m=+3.011832820 container died a4ba64c7d6e944a76f5a39b8527d482aa99f763a781b30f67edf607b25d86f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T06:41:22.570 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:41:22 smithi090 podman[74669]: 2024-02-26 06:41:22.311391678 +0000 UTC m=+3.367864242 container cleanup a4ba64c7d6e944a76f5a39b8527d482aa99f763a781b30f67edf607b25d86f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-26T06:41:22.570 INFO:journalctl@ceph.mds.h.smithi090.stdout:Feb 26 06:41:22 smithi090 bash[74669]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-h 2024-02-26T06:41:22.687 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.h.service' 2024-02-26T06:41:22.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:22.722 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-26T06:41:22.722 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-26T06:41:22.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.k 2024-02-26T06:41:22.865 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:41:22 smithi090 systemd[1]: Stopping Ceph mds.k for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:23.365 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:41:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k[66389]: 2024-02-26T06:41:23.024+0000 7f89279b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:23.365 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:41:23 smithi090 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k[66389]: 2024-02-26T06:41:23.024+0000 7f89279b1700 -1 mds.k *** got signal Terminated *** 2024-02-26T06:41:26.276 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:41:25 smithi090 podman[74790]: 2024-02-26 06:41:25.952299873 +0000 UTC m=+3.019794375 container died 4d0a8646502364846efe3a4afa9c3058b8105128943f042d52735c4f95321c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T06:41:26.568 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:41:26 smithi090 podman[74790]: 2024-02-26 06:41:26.359207405 +0000 UTC m=+3.426701899 container cleanup 4d0a8646502364846efe3a4afa9c3058b8105128943f042d52735c4f95321c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-26T06:41:26.568 INFO:journalctl@ceph.mds.k.smithi090.stdout:Feb 26 06:41:26 smithi090 bash[74790]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-k 2024-02-26T06:41:26.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.k.service' 2024-02-26T06:41:26.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:26.738 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-26T06:41:26.738 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-26T06:41:26.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.c 2024-02-26T06:41:27.062 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:41:26 smithi167 systemd[1]: Stopping Ceph mds.c for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:27.495 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:41:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c[67451]: 2024-02-26T06:41:27.062+0000 7f80a8d5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:27.495 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:41:27 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c[67451]: 2024-02-26T06:41:27.062+0000 7f80a8d5e700 -1 mds.c *** got signal Terminated *** 2024-02-26T06:41:30.245 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:41:29 smithi167 podman[74924]: 2024-02-26 06:41:29.968539867 +0000 UTC m=+3.019894470 container died a5841f2d50c2a3bbe8bb48bc99c399ff597e9554573a26dca99c9e7ff0c01348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-02-26T06:41:30.589 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:41:30 smithi167 podman[74924]: 2024-02-26 06:41:30.362677675 +0000 UTC m=+3.414032275 container cleanup a5841f2d50c2a3bbe8bb48bc99c399ff597e9554573a26dca99c9e7ff0c01348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T06:41:30.589 INFO:journalctl@ceph.mds.c.smithi167.stdout:Feb 26 06:41:30 smithi167 bash[74924]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-c 2024-02-26T06:41:30.712 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.c.service' 2024-02-26T06:41:30.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:30.751 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-26T06:41:30.752 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-26T06:41:30.752 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.f 2024-02-26T06:41:30.995 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:41:30 smithi167 systemd[1]: Stopping Ceph mds.f for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:31.495 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:41:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f[67145]: 2024-02-26T06:41:31.100+0000 7fae92f51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:31.495 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:41:31 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f[67145]: 2024-02-26T06:41:31.100+0000 7fae92f51700 -1 mds.f *** got signal Terminated *** 2024-02-26T06:41:34.245 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:41:33 smithi167 podman[75056]: 2024-02-26 06:41:33.958564657 +0000 UTC m=+2.991570918 container died bee6b06b91a6a72f7609c818fa40397c8131776b0a4b737d42aec1ee4d23780a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T06:41:34.602 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:41:34 smithi167 podman[75056]: 2024-02-26 06:41:34.335034295 +0000 UTC m=+3.368040530 container cleanup bee6b06b91a6a72f7609c818fa40397c8131776b0a4b737d42aec1ee4d23780a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:34.602 INFO:journalctl@ceph.mds.f.smithi167.stdout:Feb 26 06:41:34 smithi167 bash[75056]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-f 2024-02-26T06:41:34.761 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.f.service' 2024-02-26T06:41:34.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:34.798 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-26T06:41:34.798 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-26T06:41:34.798 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.i 2024-02-26T06:41:35.130 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:41:34 smithi167 systemd[1]: Stopping Ceph mds.i for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:35.495 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:41:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:41:35.130+0000 7f82ce5f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:35.639 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:41:35 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i[67775]: 2024-02-26T06:41:35.130+0000 7f82ce5f3700 -1 mds.i *** got signal Terminated *** 2024-02-26T06:41:38.245 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:41:37 smithi167 podman[75175]: 2024-02-26 06:41:37.967445792 +0000 UTC m=+2.955124146 container died 688601f53c498f9655e774b3eae69dacdd2500e1c961c7002109c6d51cdc41b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:38.581 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:41:38 smithi167 podman[75175]: 2024-02-26 06:41:38.322049501 +0000 UTC m=+3.309727845 container cleanup 688601f53c498f9655e774b3eae69dacdd2500e1c961c7002109c6d51cdc41b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T06:41:38.581 INFO:journalctl@ceph.mds.i.smithi167.stdout:Feb 26 06:41:38 smithi167 bash[75175]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-i 2024-02-26T06:41:38.736 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.i.service' 2024-02-26T06:41:38.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:38.771 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-26T06:41:38.771 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-26T06:41:38.772 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.l 2024-02-26T06:41:39.100 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:38 smithi167 systemd[1]: Stopping Ceph mds.l for 5e92f702-d46e-11ee-95c3-87774f69a715... 2024-02-26T06:41:39.495 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l[66837]: 2024-02-26T06:41:39.100+0000 7f80033fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T06:41:39.495 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:39 smithi167 ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l[66837]: 2024-02-26T06:41:39.100+0000 7f80033fc700 -1 mds.l *** got signal Terminated *** 2024-02-26T06:41:42.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:41 smithi167 podman[75295]: 2024-02-26 06:41:41.961278059 +0000 UTC m=+2.964144286 container died bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-26T06:41:42.683 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:42 smithi167 podman[75295]: 2024-02-26 06:41:42.268025819 +0000 UTC m=+3.270892064 container cleanup bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-02-26T06:41:42.683 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:42 smithi167 bash[75295]: ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l 2024-02-26T06:41:42.683 INFO:journalctl@ceph.mds.l.smithi167.stdout:Feb 26 06:41:42 smithi167 podman[75308]: 2024-02-26 06:41:42.468904359 +0000 UTC m=+0.507521298 container remove bf954bc3704bf1cf2f3c481a51ebaa011518b4bd88aaac6fad072e91753d74ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-5e92f702-d46e-11ee-95c3-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-02-26T06:41:42.696 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5e92f702-d46e-11ee-95c3-87774f69a715@mds.l.service' 2024-02-26T06:41:42.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T06:41:42.730 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-26T06:41:42.731 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T06:41:43.068 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:41:48.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T06:41:48.328 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:41:50.784 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T06:41:50.977 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:41:53.383 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:41:53.492 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:41:53.527 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T06:41:53.558 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-26T06:41:54.066 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi006/crash 2024-02-26T06:41:54.119 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash -- . 2024-02-26T06:41:54.232 INFO:teuthology.orchestra.run.smithi006.stderr:tar: /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T06:41:54.232 INFO:teuthology.orchestra.run.smithi006.stderr:tar: Error is not recoverable: exiting now 2024-02-26T06:41:54.238 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi090/crash 2024-02-26T06:41:54.239 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash -- . 2024-02-26T06:41:54.280 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T06:41:54.281 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-02-26T06:41:54.282 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi167/crash 2024-02-26T06:41:54.283 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash -- . 2024-02-26T06:41:54.329 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T06:41:54.329 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-02-26T06:41:54.330 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-26T06:41:54.330 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T06:41:54.441 INFO:tasks.cephadm:Compressing logs... 2024-02-26T06:41:54.441 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T06:41:54.445 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T06:41:54.448 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T06:41:54.484 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T06:41:54.484 INFO:teuthology.orchestra.run.smithi090.stderr:: No such file or directory 2024-02-26T06:41:54.484 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.10.log 2024-02-26T06:41:54.484 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T06:41:54.485 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.b.log 2024-02-26T06:41:54.485 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.4.log 2024-02-26T06:41:54.485 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.1.log 2024-02-26T06:41:54.485 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T06:41:54.486 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.k.log 2024-02-26T06:41:54.486 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.h.log 2024-02-26T06:41:54.486 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.7.log 2024-02-26T06:41:54.486 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.e.log 2024-02-26T06:41:54.487 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.7.log: 90.1% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T06:41:54.487 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.y.log 2024-02-26T06:41:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T06:41:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2024-02-26T06:41:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.l.log 2024-02-26T06:41:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.i.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.11.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.z.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.i.log: /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.11.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.8.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.5.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.f.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T06:41:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.f.log: 90.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log: 83.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.2.log 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log: 88.5% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.c.log 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.2.log: 91.9% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.z.log.gz 2024-02-26T06:41:54.491 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T06:41:54.494 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T06:41:54.496 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.c.log 2024-02-26T06:41:54.501 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log 2024-02-26T06:41:54.508 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T06:41:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log: 83.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T06:41:54.512 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% 91.9% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.y.log.gz 2024-02-26T06:41:54.513 INFO:teuthology.orchestra.run.smithi006.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T06:41:54.513 INFO:teuthology.orchestra.run.smithi006.stderr:: No such file or directory 2024-02-26T06:41:54.514 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.6.log 2024-02-26T06:41:54.515 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.a.log 2024-02-26T06:41:54.516 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T06:41:54.517 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T06:41:54.517 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.b.log 2024-02-26T06:41:54.518 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.g.log 2024-02-26T06:41:54.518 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.j.log 2024-02-26T06:41:54.520 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.3.log 2024-02-26T06:41:54.521 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.x.log 2024-02-26T06:41:54.522 INFO:teuthology.orchestra.run.smithi006.stderr: 90.0% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T06:41:54.523 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.0.log 2024-02-26T06:41:54.526 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.x.log: /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T06:41:54.526 INFO:teuthology.orchestra.run.smithi006.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T06:41:54.527 INFO:teuthology.orchestra.run.smithi006.stderr: 94.4% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.j.log.gz 2024-02-26T06:41:54.527 INFO:teuthology.orchestra.run.smithi006.stderr: 94.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.g.log.gz 2024-02-26T06:41:54.529 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T06:41:54.529 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log 2024-02-26T06:41:54.530 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T06:41:54.532 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.9.log 2024-02-26T06:41:54.535 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.d.log 2024-02-26T06:41:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T06:41:54.537 INFO:teuthology.orchestra.run.smithi006.stderr: 88.4% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T06:41:54.541 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.a.log 2024-02-26T06:41:54.543 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T06:41:54.548 INFO:teuthology.orchestra.run.smithi090.stderr: 94.4% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.k.log.gz 2024-02-26T06:41:54.550 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.a.log: 90.8% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T06:41:54.572 INFO:teuthology.orchestra.run.smithi090.stderr: 94.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.e.log.gz 2024-02-26T06:41:54.578 INFO:teuthology.orchestra.run.smithi090.stderr: 93.6% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.h.log.gz 2024-02-26T06:41:54.636 INFO:teuthology.orchestra.run.smithi167.stderr: 94.4% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.f.log.gz 2024-02-26T06:41:54.697 INFO:teuthology.orchestra.run.smithi167.stderr: 95.0% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.l.log.gz 2024-02-26T06:41:54.720 INFO:teuthology.orchestra.run.smithi006.stderr: 93.8% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.a.log.gz 2024-02-26T06:41:54.892 INFO:teuthology.orchestra.run.smithi006.stderr: 89.9% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mgr.x.log.gz 2024-02-26T06:41:54.954 INFO:teuthology.orchestra.run.smithi167.stderr: 91.9% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.c.log.gz 2024-02-26T06:41:55.110 INFO:teuthology.orchestra.run.smithi090.stderr: 91.8% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.b.log.gz 2024-02-26T06:41:55.221 INFO:teuthology.orchestra.run.smithi090.stderr: 93.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.4.log.gz 2024-02-26T06:41:55.407 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.1.log.gz 2024-02-26T06:41:55.453 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.b.log.gz 2024-02-26T06:41:55.514 INFO:teuthology.orchestra.run.smithi167.stderr: 93.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.2.log.gz 2024-02-26T06:41:55.568 INFO:teuthology.orchestra.run.smithi006.stderr: 93.1% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.9.log.gz 2024-02-26T06:41:55.620 INFO:teuthology.orchestra.run.smithi006.stderr: 93.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.0.log.gz 2024-02-26T06:41:55.703 INFO:teuthology.orchestra.run.smithi090.stderr: 93.4% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.7.log.gz 2024-02-26T06:41:55.728 INFO:teuthology.orchestra.run.smithi167.stderr: 93.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.5.log.gz 2024-02-26T06:41:55.755 INFO:teuthology.orchestra.run.smithi167.stderr: 93.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.8.log.gz 2024-02-26T06:41:55.767 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.10.log.gz 2024-02-26T06:41:55.769 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-02-26T06:41:55.770 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m1.307s 2024-02-26T06:41:55.770 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m5.787s 2024-02-26T06:41:55.770 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m0.223s 2024-02-26T06:41:55.776 INFO:teuthology.orchestra.run.smithi006.stderr: 90.5% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mon.a.log.gz 2024-02-26T06:41:55.788 INFO:teuthology.orchestra.run.smithi167.stderr: 93.6% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.11.log.gz 2024-02-26T06:41:55.931 INFO:teuthology.orchestra.run.smithi006.stderr: 93.1% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.6.log.gz 2024-02-26T06:41:55.956 INFO:teuthology.orchestra.run.smithi006.stderr: 95.4% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.d.log.gz 2024-02-26T06:41:55.999 INFO:teuthology.orchestra.run.smithi006.stderr: 93.3% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-osd.3.log.gz 2024-02-26T06:41:56.003 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-02-26T06:41:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m1.526s 2024-02-26T06:41:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:user 0m7.713s 2024-02-26T06:41:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m0.662s 2024-02-26T06:41:56.136 INFO:teuthology.orchestra.run.smithi167.stderr: 94.2% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.c.log.gz 2024-02-26T06:42:04.685 INFO:teuthology.orchestra.run.smithi167.stderr: 93.0% -- replaced with /var/log/ceph/5e92f702-d46e-11ee-95c3-87774f69a715/ceph-mds.i.log.gz 2024-02-26T06:42:04.686 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-02-26T06:42:04.686 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m10.220s 2024-02-26T06:42:04.686 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m16.852s 2024-02-26T06:42:04.686 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m0.574s 2024-02-26T06:42:04.687 INFO:tasks.cephadm:Archiving logs... 2024-02-26T06:42:04.688 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi006/log 2024-02-26T06:42:04.689 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T06:42:05.757 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi090/log 2024-02-26T06:42:05.758 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T06:42:06.653 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi167/log 2024-02-26T06:42:06.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T06:42:09.095 INFO:tasks.cephadm:Removing cluster... 2024-02-26T06:42:09.095 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --force 2024-02-26T06:42:09.438 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:42:10.083 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --force 2024-02-26T06:42:10.275 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:42:10.535 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5e92f702-d46e-11ee-95c3-87774f69a715 --force 2024-02-26T06:42:10.729 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 5e92f702-d46e-11ee-95c3-87774f69a715 2024-02-26T06:42:11.001 INFO:tasks.cephadm:Removing cephadm ... 2024-02-26T06:42:11.001 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T06:42:11.060 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T06:42:11.076 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T06:42:11.091 INFO:tasks.cephadm:Teardown complete 2024-02-26T06:42:11.092 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-26T06:42:11.103 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-26T06:42:11.104 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T06:42:11.111 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T06:42:11.118 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T06:42:11.276 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-26T06:42:11.287 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-26T06:42:11.287 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:42:11.295 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:42:11.298 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T06:42:11.310 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:42:11.312 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:42:11.314 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:42:11.315 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-02-26T06:42:11.315 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +743us[ +757us] +/- 127ms 2024-02-26T06:42:11.315 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -591us[ -577us] +/- 76ms 2024-02-26T06:42:11.315 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +143us[ +157us] +/- 65ms 2024-02-26T06:42:11.315 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:42:11.317 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:42:11.317 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-02-26T06:42:11.317 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +776us[ +776us] +/- 127ms 2024-02-26T06:42:11.317 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv02.front.sepia.ceph.com 3 7 377 111 -544us[ -501us] +/- 75ms 2024-02-26T06:42:11.317 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +171us[ +171us] +/- 65ms 2024-02-26T06:42:11.318 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:42:11.344 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-02-26T06:42:11.355 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T06:42:11.355 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-02-26T06:42:11.355 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 +811us[ +811us] +/- 127ms 2024-02-26T06:42:11.355 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv02.front.sepia.ceph.com 3 8 377 174 -489us[ -487us] +/- 74ms 2024-02-26T06:42:11.355 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +208us[ +208us] +/- 65ms 2024-02-26T06:42:11.356 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T06:42:11.357 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-26T06:42:11.368 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-26T06:42:11.369 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-26T06:42:11.384 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T06:42:11.641 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T06:42:11.716 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T06:42:11.789 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:42:11.897 INFO:teuthology.orchestra.run.smithi006.stdout:type=AVC msg=audit(1708927331.282:199): avc: denied { checkpoint_restore } for pid=1332 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-02-26T06:42:11.899 DEBUG:teuthology.task.selinux:ubuntu@smithi006.front.sepia.ceph.com has 1 denials 2024-02-26T06:42:11.900 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:42:11.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:42:11.931 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T06:42:11.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:42:11.962 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi006.front.sepia.ceph.com: ['type=AVC msg=audit(1708927331.282:199): avc: denied { checkpoint_restore } for pid=1332 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-02-26T06:42:12.007 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-26T06:42:12.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-26T06:42:12.053 INFO:teuthology.task.internal:Duration was 2404.024741 seconds 2024-02-26T06:42:12.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-26T06:42:12.084 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-26T06:42:12.085 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T06:42:12.099 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T06:42:12.103 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T06:42:12.144 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:42:12.146 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:42:12.270 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T06:42:12.751 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-26T06:42:12.751 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-02-26T06:42:12.752 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T06:42:12.814 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-02-26T06:42:12.814 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T06:42:12.840 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-02-26T06:42:12.840 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T06:42:12.872 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-26T06:42:12.872 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T06:42:12.880 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T06:42:12.883 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T06:42:12.979 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-26T06:42:12.980 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T06:42:12.985 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T06:42:12.988 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T06:42:13.573 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-26T06:42:13.584 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-26T06:42:13.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T06:42:13.684 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T06:42:13.713 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T06:42:13.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-26T06:42:13.752 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:42:13.760 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:42:13.762 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:42:13.786 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-02-26T06:42:13.806 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-02-26T06:42:13.865 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-02-26T06:42:13.913 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:42:13.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:42:13.963 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:42:13.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:42:13.977 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T06:42:13.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T06:42:13.991 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-26T06:42:14.003 INFO:teuthology.task.internal:Transferring archived files... 2024-02-26T06:42:14.004 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi006 2024-02-26T06:42:14.004 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T06:42:14.125 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi090 2024-02-26T06:42:14.126 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T06:42:14.165 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427/remote/smithi167 2024-02-26T06:42:14.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T06:42:14.205 INFO:teuthology.task.internal:Removing archive directory... 2024-02-26T06:42:14.205 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T06:42:14.213 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T06:42:14.215 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T06:42:14.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-26T06:42:14.282 INFO:teuthology.task.internal:Not uploading archives. 2024-02-26T06:42:14.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-26T06:42:14.303 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-26T06:42:14.303 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T06:42:14.311 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T06:42:14.313 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T06:42:14.325 INFO:teuthology.orchestra.run.smithi090.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 06:42 /home/ubuntu/cephtest 2024-02-26T06:42:14.327 INFO:teuthology.orchestra.run.smithi167.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 06:42 /home/ubuntu/cephtest 2024-02-26T06:42:14.364 INFO:teuthology.orchestra.run.smithi006.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 06:42 /home/ubuntu/cephtest 2024-02-26T06:42:14.367 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-26T06:42:14.380 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-26T06:42:14.403 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi006.front.sepia.ceph.com: ['type=AVC msg=audit(1708927331.282:199): avc: denied { checkpoint_restore } for pid=1332 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-02-26T06:42:14.404 INFO:teuthology.nuke:Checking targets against current locks 2024-02-26T06:42:14.428 DEBUG:teuthology.nuke:shortname: smithi006 2024-02-26T06:42:14.429 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T06:42:14.448 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:52:17.533602', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZPdkDZSRXwvoUvPcwAYAldqb0CHJhFpgKlOxRkyHNF6rbw1E2/xgFSq4EqU7IaDXqrmGdkezUdpgdkbmhtm9I='} 2024-02-26T06:42:14.449 DEBUG:teuthology.nuke:shortname: smithi090 2024-02-26T06:42:14.450 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T06:42:14.475 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:52:17.532513', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCWZElmQG/kXQ7G++W9esi9RidN6ruebCJTuqMKCH6GTXUldq95uZ52llHODIcBZYdrfxfYQpcyH1IpskP5OiY='} 2024-02-26T06:42:14.477 DEBUG:teuthology.nuke:shortname: smithi167 2024-02-26T06:42:14.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T06:42:14.514 INFO:teuthology.orchestra.console.smithi006:Power off 2024-02-26T06:42:14.515 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T06:42:14.525 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_05:44:42-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573427', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 05:52:17.531197', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJN6FQPyU3rQNmlN7XjX9ozDAbCOMd7C4QTI8TsxNvfiK+XiywOGT8wepALDNcH6c3vpDYA/5J2Lik6ezL6wxo='} 2024-02-26T06:42:14.539 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2024-02-26T06:42:14.539 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:14.561 INFO:teuthology.orchestra.console.smithi090:Power off 2024-02-26T06:42:14.561 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T06:42:14.572 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-02-26T06:42:14.585 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-02-26T06:42:14.585 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:14.601 INFO:teuthology.orchestra.console.smithi167:Power off 2024-02-26T06:42:14.601 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T06:42:14.610 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-02-26T06:42:14.626 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2024-02-26T06:42:14.626 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:14.651 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-02-26T06:42:18.573 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:18.611 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:18.652 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:18.685 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-02-26T06:42:18.721 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-02-26T06:42:18.767 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-02-26T06:42:22.688 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:22.722 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:22.768 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:22.801 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-02-26T06:42:22.834 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-02-26T06:42:22.880 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-02-26T06:42:26.806 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:26.835 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:26.881 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T06:42:26.917 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2024-02-26T06:42:26.944 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-02-26T06:42:26.989 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-02-26T06:42:27.018 INFO:teuthology.orchestra.console.smithi006:Power off completed 2024-02-26T06:42:27.045 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-02-26T06:42:27.089 INFO:teuthology.orchestra.console.smithi167:Power off completed 2024-02-26T06:42:27.196 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} duration: 2404.024740934372 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi006 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ba6fbbfee45488c6bcf0f2f12a3da33f007f0031 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=948a37067c4e4025b3c8fa9ef7475912 status: fail success: false 2024-02-26T06:42:27.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T06:42:27.286 INFO:teuthology.run:FAIL